From nobody Wed Mar 19 16:34:00 2025 X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4ZHvRZ4wqgz5rMd7 for ; Wed, 19 Mar 2025 16:34:02 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4ZHvRZ4Pb8z3rNb; Wed, 19 Mar 2025 16:34:02 +0000 (UTC) (envelope-from kevans@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1742402042; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JFNYRsMW/gDVy9Yo15i9H0aHyoJUWGATm/hkmCr6c3A=; b=hl9XGgPL/dIVFAhRiVu6Rss2cfMt1vK7it6iAuUBiKeJEzSSPZVRl1wjvmVHBTIL2QOAh9 Lm+z2espmiUlLsIf8a24CkbUViLyY+VT00Kbg3KNvIYGSgcJ+MOwjqdqO/0yQijMdaD35u J665fpScb55eDiXpJ8tt6B7z8G8pHrj2MhxHWkoDa3YbNSqtMaxvPgpmPeyEgYCumwNd0/ /Ie/qaYlUwKfeisQERhq7VqeasXldJUJ8Eo1v+BpN0rTl0uRs1NnwJag1mv/MvtgjRnN+r 01uBKjUAIFD/Alm/B68OkvLXoBQElniKm0RCS3nTuXHLsGSpHPyaO2C/c3xGbA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1742402042; a=rsa-sha256; cv=none; b=b6e/JhTbRcn1J7wiHod06TIgNGqFjAWDInv0xGaAnc0q4LUZObQHnr0ahskdtFnnF2V/ic vkJB3R8T6XKWUnaejd1ODd6sRtunaqhnAqLorgDbkRqnggQC9fhy2ToayreovGra+fVGEy 06pguPcLSzCFN+GuOG7492ux6YFIPdcl9e1oCAevtqR0E85nJk7lQW56NrVs3Z8j6YlGTy 7T5RUnWVWgM3flRAXjqm2BdCmoGdeauYMhjmRcca+mSJMhEOmBTcZKqVy4H1qTN1/vEB5p xlhI/xVqVXhMMJnDyGVcxR0tRodeR/Js4kbxjkBGijZT9oUI30g80VYZUTpC8w== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1742402042; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JFNYRsMW/gDVy9Yo15i9H0aHyoJUWGATm/hkmCr6c3A=; b=UjDhVi/oROIEAh6fjXGYlzs7e9Es7qhVOB8RiPFZmrL61pU8Rl0zJ+n+N/V87IMO1BinVL Q9FOOZcOZS0dXRZZXoUeuzTDwUJbBGBOsfb7HPkAtbs9qWfi86zQRp/PDs6G2+/9jpM5N1 ugyprxMO5UM58wz/iQfEAG1rwOuFyifDb+43g89Cn9xlFZ9vHvcGljvGfZlXJ+5rHYuNQI ns5F6AEQg/DGFTueXd8HV5P+1oJapRJCP/NpsjLe9pM4XwCKeAylgGi/5tBKPN//ljNDg/ N81e9Iuf7HiZc+u6UjWOS8pgOmRhquTbhzRJlqGQV7pB4p/I9+KPp5K3oBkWKg== Received: from [10.9.4.95] (unknown [209.182.120.176]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: kevans/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4ZHvRZ1WSJznTB; Wed, 19 Mar 2025 16:34:02 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Message-ID: <29fe13ff-decc-456e-8afb-031de496932d@FreeBSD.org> Date: Wed, 19 Mar 2025 11:34:00 -0500 List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: panic: tcp_do_segment: sent too much To: Michael Tuexen Cc: freebsd-net@freebsd.org, Gleb Smirnoff References: <020C5409-C587-4D06-B764-1AD0C5783AB2@lurchi.franken.de> Content-Language: en-US From: Kyle Evans In-Reply-To: <020C5409-C587-4D06-B764-1AD0C5783AB2@lurchi.franken.de> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit On 1/13/25 13:01, Michael Tuexen wrote: >> On 13. Jan 2025, at 02:44, Kyle Evans wrote: >> >> Hi, >> >> I seem to have hit this panic during normal laptop usage with a -CURRENT running at 2832af7b4ea256b18ef4dbf2ff97a50765f0609a: >> >> panic: tcp_do_segment: sent too much >> cpuid = 5 >> time = 1736731772 >> KDB: stack backtrace: >> db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe015b74b9b0 >> vpanic() at vpanic+0x136/frame 0xfffffe015b74bae0 >> panic() at panic+0x43/frame 0xfffffe015b74bb40 >> tcp_do_segment() at tcp_do_segment+0x2998/frame 0xfffffe015b74bc20 >> tcp_input_with_port() at tcp_input_with_port+0x10f8/frame 0xfffffe015b74bd70 >> tcp_input() at tcp_input+0xb/frame 0xfffffe015b74bd80 >> ip_input() at ip_input+0x28f/frame 0xfffffe015b74bde0 >> swi_net() at swi_net+0x19b/frame 0xfffffe015b74be60 >> ithread_loop() at ithread_loop+0x266/frame 0xfffffe015b74bef0 >> fork_exit() at fork_exit+0x82/frame 0xfffffe015b74bf30 >> fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe015b74bf30 >> >> Doing a little bit of inspection: >> >> (kgdb) print tp->t_dupacks >> $1 = 1 >> (kgdb) print tp->snd_limited >> $6 = 0 '\000' >> (kgdb) p/x tp->t_flags >> $4 = 0x10000264 >> (kgdb) print maxseg >> $1 = 1428 >> >> `sent` appears to be optimized out. As far as I know, I don't hit this one often, but I'm wondering if there's anything else useful I could extract here / if anyone has a theory about what happened. > Hi Kyle, > > you can run your system with TCP black box logging enabled by using > > sysctl net.inet.tcp.bb.log_auto_all=1 > sysctl net.inet.tcp.bb.log_auto_ratio=1 > sysctl net.inet.tcp.bb.log_auto_mode=1 > > Also have the tcplog_dumper running. You can use > > tcplog_dumper_enable="YES" > > in /etc/rc.conf > > You can install tcplog_dumper by > > pkg intall tcplog_dumper > > Using these settings (mode = TCP_LOG_STATE_TAIL), keeps the trailing events of all TCP connections, but does not dump anything to disk. If the kernel panics, we can get the BBLog information from the core. > > Please note that Gleb has just committed two fixes which also address a panic() like the one you are experiencing. > I've seen another of these on a tree branched from early March @ d4c81623accede ("universe: Update the default GCC toolchain to GCC 14") with the above enabled; what can I provide, or has another likely cause been addressed since then? Thanks, Kyle Evans