TCP hostcache inappropriately limits MSS
Date: Tue, 01 Sep 2026 14:57:30 UTC
hello,
i'm testing between a FreeBSD 15.1 system with a cxl(4):
cxl0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 9000
and a Linux system with an ix(4):
3: enp37s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP group default qlen 1000
i purge the TCP hostcache, then start iperf3 server on the FreeBSD system:
hemlock% iperf3 -s
and connect from the client:
ilythia% iperf3 -c hemlock
everything looks fine:
15:48:57.122710 IP6 ilythia.le-fay.org.37886 > hemlock.le-fay.org.5201:
Flags [S], seq 4236345934, win 62580,
options [mss 8940,sackOK,TS val 3473128328 ecr 0,nop,wscale 10], length 0
15:48:57.122766 IP6 hemlock.le-fay.org.5201 > ilythia.le-fay.org.37886:
Flags [S.], seq 3257798300, ack 4236345935, win 65535,
options [mss 8940,nop,wscale 14,sackOK,TS val 265287490 ecr 3473128328], length 0
then i connect from the client in "reverse" mode:
ilythia% iperf3 -Rc hemlock
now the server limits the MSS to 1440:
15:50:21.828881 IP6 ilythia.le-fay.org.34150 > hemlock.le-fay.org.5201:
Flags [S], seq 4215154639, win 62580,
options [mss 8940,sackOK,TS val 3887087251 ecr 0,nop,wscale 10], length 0
15:50:21.828937 IP6 hemlock.le-fay.org.5201 > ilythia.le-fay.org.34150:
Flags [S.], seq 2707634511, ack 4215154640, win 65535,
options [mss 1440,nop,wscale 14,sackOK,TS val 1253750990 ecr 3887087251], length 0
this problem seems a bit non-deterministic; it might take a few tries
for the MSS to drop, but it always does eventually, unless i disable
the TCP hostcache, in which case everything works fine.
this feels like a bug. the L3 path MTU is 9000 bytes, so FreeBSD should
always accept and use 8940 as the MSS.
in the broken state, the hostcache entry for the iperf3 client looks like this:
IP address MTU SSTRESH RTT RTTVAR CWND SENDPIPE RECVPIPE EXP
2001:8b0:aab5:c003::2 1500 30750 6ms 8ms 26600 0 0 3600