From nobody Thu Jun 05 05:04:44 2025 X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4bCXSG6TYcz5yCQr; Thu, 05 Jun 2025 05:04:46 +0000 (UTC) (envelope-from tuexen@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4bCXSG5y1pz45gX; Thu, 05 Jun 2025 05:04:46 +0000 (UTC) (envelope-from tuexen@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1749099886; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WauvaLw3EPrGqEIC5I7UjSVNofTSFyF9kGg9IeC849o=; b=LrsZEZvzNkgJQ6DtBHZJZUwawu7/osStJjNDiQfB5M//+tRc/LRM65OfEtriMwGypLBA0J 6WArxwf4V7rv8+c3dPQqkrKR5KYute4jivnzSMcS2t1wH5rjt5Gk64F93XvXy5VGn4hadd viD0QVHMvdPke6QNMauAhfIcRn9P2B5FkrBlGg7UGsYmtyXnUI4GXsi63CGjcmm7J9qKNd KD5LvejK7S682xS/O6CR373RMECPkCd7QyJYpIQmlpDelbCs8AWWOXGMrCzTIc1fOWAnFc f7ZQa35GoWTJLnbc2M/pYW4NEWgxIYUC6LAxcBcFP7SYTTNwdCe+cv61Wpgn+Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1749099886; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WauvaLw3EPrGqEIC5I7UjSVNofTSFyF9kGg9IeC849o=; b=NhkD7NPMgpVRLQDcI1AK5x5aUacYxr+RGv7zUqUGI79WASDxzzyR0mNqv56uRyEGHk1dPP 15OqpV6pkUGLKUVeeLpf6AYnIAZNny7KS9/LjYQnNGmkjaUkBGl95BjtFMRTBRx2Ejk1dH pGcZOgn+qjH+tTFU0N7DS+CPWI5ROf3jfxcW5BdoDJgKyzNp1sDphg/x5CieGmGriHjxGu 1leiJohJ0gsjMDcft1mCG+RXUNaREOsk8gYOSj507PtkkdvWLT+JIODIrDs3VWmIYazBj1 sxk1q6XIL34ppZZ4XfQ4RBjO8WjEpd8RNrlK/0bdKxXZFwMkB/xic2IC8HxdmA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1749099886; a=rsa-sha256; cv=none; b=MyVorMI3ckqKRLnFtgMmzV9nvl5bjejT5uDG2k6z4DH2u2rFLGoenPU6GGqRXUZyrB8k9P fBXm0fAEUBDstXzmZDawYMGC0MQ5A2U8HiYv1op+37X2qoimqMEwrPbc22fNOp5ZtQ13TQ eIarEi3ebmIaFNuLbbxL3vhzUZcxK5hMOHd3YIH8SSAd/h5v1ftXaADPLzlx0W5CHw8w4H svSwJRlGeMBPlVDH9IXRrdo/ZPPhelTr1rZuGpoJ1LEzwAP+uuUgAEt1WJD4O0pah9xq2Z 24AYiyy1LExpZncZR9UC8kfKHWGc5P2ok30Jx2v2Zzm7iEssUxor7sz9EjlmnA== Received: from smtpclient.apple (unknown [IPv6:2a02:8109:1101:be00:fc66:3001:3f63:4163]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: tuexen) by smtp.freebsd.org (Postfix) with ESMTPSA id 4bCXSG2GM0zv3c; Thu, 05 Jun 2025 05:04:46 +0000 (UTC) (envelope-from tuexen@FreeBSD.org) Content-Type: text/plain; charset=utf-8 List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3826.600.51.1.1\)) Subject: Re: Problem with net.inet.tcp.path_mtu_discovery=1 From: Michael Tuexen In-Reply-To: Date: Thu, 5 Jun 2025 07:04:44 +0200 Cc: questions@freebsd.org, freebsd-net Content-Transfer-Encoding: quoted-printable Message-Id: <2954346E-B540-4E36-AA34-7FB08935C350@FreeBSD.org> References: <9728060D-2C02-426B-BACE-F2D2F651A62F@cretaforce.gr> To: Christos Chatzaras X-Mailer: Apple Mail (2.3826.600.51.1.1) > On 4. Jun 2025, at 19:29, Christos Chatzaras = wrote: >=20 >=20 >=20 >> On 4 Jun 2025, at 19:36, Dave Cottlehuber wrote: >>=20 >> On Wed, 4 Jun 2025, at 16:36, Christos Chatzaras wrote: >>> Hello, >>>=20 >>> I manage some servers hosting websites. >>=20 >> What does tcpdump/wireshark show for traffic, particularly icmp? = Wireshark is very helpful in explaining some issues. >>=20 >> What is the actual MTU on the working net vs the failing one? >>=20 >> Is there a local MTU where the failing websites start working again? >>=20 >> see ping(8) and use -v -D -s =E2=80=A6. together to find a working = MTU and cross check with tcpdump to find where things seem to break. >>=20 >> On a recent cloud environment I needed to add =E2=80=98 set = reassemble yes no-df=E2=80=99 to my pf.conf to address MTU issues = between VNET jails and the internet. >>=20 >> Happy hunting >> Dave >>=20 >=20 > First, I reverted the server settings to their defaults: > sysctl net.inet.tcp.path_mtu_discovery=3D1 > sysctl net.inet.tcp.pmtud_blackhole_detection=3D0 >=20 > Next, I set the MTU on my local computer to 1460 and everything worked = as expected: > tcpdump: listening on en0, link-type EN10MB (Ethernet), snapshot = length 524288 bytes > 20:15:05.651375 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto = TCP (6), length 64) > 192.168.2.18.65322 > 94.130.217.87.443: Flags [S], cksum 0x293e = (correct), seq 3503095669, win 65535, options [mss 1420,nop,wscale = 6,nop,nop,TS val 639376397 ecr 0,sackOK,eol], length 0 > 20:15:05.705913 IP (tos 0x0, ttl 54, id 0, offset 0, flags [DF], proto = TCP (6), length 60) > 94.130.217.87.443 > 192.168.2.18.65322: Flags [S.], cksum 0x9c22 = (correct), seq 3647364942, ack 3503095670, win 65535, options [mss = 1460,nop,wscale 6,sackOK,TS val 1782053626 ecr 639376397], length 0 >=20 > However, when I set my local computer=E2=80=99s MTU back to 1500 (the = default), the issue reappeared: > tcpdump: listening on en0, link-type EN10MB (Ethernet), snapshot = length 524288 bytes > 20:17:45.662993 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto = TCP (6), length 64) > 192.168.2.18.65333 > 94.130.217.87.443: Flags [S], cksum 0x4a07 = (correct), seq 3674289142, win 65535, options [mss 1460,nop,wscale = 6,nop,nop,TS val 681359835 ecr 0,sackOK,eol], length 0 > 20:17:45.726988 IP (tos 0x0, ttl 54, id 0, offset 0, flags [DF], proto = TCP (6), length 60) > 94.130.217.87.443 > 192.168.2.18.65333: Flags [S.], cksum 0x9b1d = (correct), seq 1443843488, ack 3674289143, win 65535, options [mss = 1460,nop,wscale 6,sackOK,TS val 2890559459 ecr 681359835], length 0 >=20 > So, with local computer MTU 1460, everything works, but with MTU 1500, = the problem persists. The difference is that you announce a smaller MSS in SYN segment you sent. This means that the peer can only send you smaller TCP segments. So there seems to be a problem if the peer sends too large TCP segments. That means that the peer must do PMTUD or TCP blackhole detection, not the local node. Best regards Michael