Re: TCP hostcache inappropriately limits MSS

> On 1. Sep 2026, at 16:57, Lexi Winter <ivy@FreeBSD.org> wrote:
> 
> hello,
> 
> i'm testing between a FreeBSD 15.1 system with a cxl(4):
> 
>  cxl0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 9000
> 
> and a Linux system with an ix(4):
> 
>  3: enp37s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP group default qlen 1000
> 
> i purge the TCP hostcache, then start iperf3 server on the FreeBSD system:
> 
>  hemlock% iperf3 -s
> 
> and connect from the client:
> 
>  ilythia% iperf3 -c hemlock
> 
> everything looks fine:
> 
>  15:48:57.122710 IP6 ilythia.le-fay.org.37886 > hemlock.le-fay.org.5201:
>    Flags [S], seq 4236345934, win 62580,
>    options [mss 8940,sackOK,TS val 3473128328 ecr 0,nop,wscale 10], length 0
>  15:48:57.122766 IP6 hemlock.le-fay.org.5201 > ilythia.le-fay.org.37886:
>    Flags [S.], seq 3257798300, ack 4236345935, win 65535,
>    options [mss 8940,nop,wscale 14,sackOK,TS val 265287490 ecr 3473128328], length 0
> 
> then i connect from the client in "reverse" mode:
> 
>  ilythia% iperf3 -Rc hemlock
> 
> now the server limits the MSS to 1440:
> 
>  15:50:21.828881 IP6 ilythia.le-fay.org.34150 > hemlock.le-fay.org.5201:
>    Flags [S], seq 4215154639, win 62580,
>    options [mss 8940,sackOK,TS val 3887087251 ecr 0,nop,wscale 10], length 0
>  15:50:21.828937 IP6 hemlock.le-fay.org.5201 > ilythia.le-fay.org.34150:
>    Flags [S.], seq 2707634511, ack 4215154640, win 65535,
>    options [mss 1440,nop,wscale 14,sackOK,TS val 1253750990 ecr 3887087251], length 0
> 
> this problem seems a bit non-deterministic; it might take a few tries
> for the MSS to drop, but it always does eventually, unless i disable
> the TCP hostcache, in which case everything works fine.
> 
> this feels like a bug.  the L3 path MTU is 9000 bytes, so FreeBSD should
> always accept and use 8940 as the MSS.
Hi Lexi,

I agree, this looks like a bug. Are both systems directly connected via a cable
or are some additional nodes (routers, switches, middleboxes) involved?

Best regards
Michael
> 
> in the broken state, the hostcache entry for the iperf3 client looks like this:
> 
>  IP address             MTU  SSTRESH  RTT   RTTVAR     CWND SENDPIPE RECVPIPE EXP
>  2001:8b0:aab5:c003::2  1500 30750    6ms      8ms    26600        0        0 3600