Re: Too aggressive TCP ACKs

From: Michael Tuexen <michael.tuexen_at_lurchi.franken.de>
Date: Fri, 21 Oct 2022 14:34:04 UTC
> On 21. Oct 2022, at 16:19, Zhenlei Huang <zlei.huang@gmail.com> wrote:
> 
> Hi,
> 
> While I was repeating https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=258755, I observed a
> strange behavior. The TCP ACKs from FreeBSD host are too aggressive.
> 
> My setup is simple:
>          A                                 B
>    [ MacOS ]  <====> [ FreeBSD VM ]
> 192.168.120.1            192.168.12.134 (disable tso and lro)
> While A <--- B, i.e. A as server and B as client, the packets rate looks good.
> 
> One session on B:
> 
> root@:~ # iperf3 -c 192.168.120.1 -b 10m
> Connecting to host 192.168.120.1, port 5201
> [  5] local 192.168.120.134 port 54459 connected to 192.168.120.1 port 5201
> [ ID] Interval           Transfer     Bitrate         Retr  Cwnd
> [  5]   0.00-1.00   sec  1.25 MBytes  10.5 Mbits/sec    0    257 KBytes       
> [  5]   1.00-2.00   sec  1.25 MBytes  10.5 Mbits/sec    0    257 KBytes       
> [  5]   2.00-3.00   sec  1.12 MBytes  9.44 Mbits/sec    0    257 KBytes       
> [  5]   3.00-4.00   sec  1.25 MBytes  10.5 Mbits/sec    0    257 KBytes       
> [  5]   4.00-5.00   sec  1.12 MBytes  9.44 Mbits/sec    0    257 KBytes       
> [  5]   5.00-6.00   sec  1.25 MBytes  10.5 Mbits/sec    0    257 KBytes       
> [  5]   6.00-7.00   sec  1.12 MBytes  9.44 Mbits/sec    0    257 KBytes       
> [  5]   7.00-8.00   sec  1.25 MBytes  10.5 Mbits/sec    0    257 KBytes       
> [  5]   8.00-9.00   sec  1.12 MBytes  9.44 Mbits/sec    0    257 KBytes       
> [  5]   9.00-10.00  sec  1.25 MBytes  10.5 Mbits/sec    0    257 KBytes       
> - - - - - - - - - - - - - - - - - - - - - - - - -
> [ ID] Interval           Transfer     Bitrate         Retr
> [  5]   0.00-10.00  sec  12.0 MBytes  10.1 Mbits/sec    0             sender
> [  5]   0.00-10.00  sec  12.0 MBytes  10.1 Mbits/sec                  receiver
> 
> iperf Done.
> 
> Another session on B:
> 
> root@:~ # netstat -w 1 -I vmx0
>             input           vmx0           output
>    packets  errs idrops      bytes    packets  errs      bytes colls
>          0     0     0          0          0     0          0     0
>          0     0     0          0          0     0          0     0
>        342     0     0      22600        526     0     775724     0
>        150     0     0       9900        851     0    1281454     0
>        109     0     0       7194        901     0    1357850     0
>        126     0     0       8316        828     0    1246632     0
>        122     0     0       8052        910     0    1370780     0
>        109     0     0       7194        819     0    1233702     0
>        120     0     0       7920        910     0    1370780     0
>        110     0     0       7260        819     0    1233702     0
>        123     0     0       8118        910     0    1370780     0
>        109     0     0       7194        819     0    1233702     0
>         73     0     0       5088        465     0     686342     0
>          0     0     0          0          0     0          0     0
>          0     0     0          0          0     0          0     0
> 
> 
> 
> ================================================================
> 
> 
> While A ---> B, i.e. A as client and B as server, the ACKs sent from B looks strange.
> 
> Session on A:
> 
> % iperf3 -c 192.168.120.134 -b 10m
> Connecting to host 192.168.120.134, port 5201
> [  5] local 192.168.120.1 port 52370 connected to 192.168.120.134 port 5201
> [ ID] Interval           Transfer     Bitrate
> [  5]   0.00-1.00   sec  1.25 MBytes  10.5 Mbits/sec                  
> [  5]   1.00-2.00   sec  1.25 MBytes  10.5 Mbits/sec                  
> [  5]   2.00-3.00   sec  1.12 MBytes  9.44 Mbits/sec                  
> [  5]   3.00-4.00   sec  1.25 MBytes  10.5 Mbits/sec                  
> [  5]   4.00-5.00   sec  1.12 MBytes  9.44 Mbits/sec                  
> [  5]   5.00-6.00   sec  1.25 MBytes  10.5 Mbits/sec                  
> [  5]   6.00-7.00   sec  1.12 MBytes  9.44 Mbits/sec                  
> [  5]   7.00-8.00   sec  1.25 MBytes  10.5 Mbits/sec                  
> [  5]   8.00-9.00   sec  1.12 MBytes  9.44 Mbits/sec                  
> [  5]   9.00-10.00  sec  1.25 MBytes  10.5 Mbits/sec                  
> - - - - - - - - - - - - - - - - - - - - - - - - -
> [ ID] Interval           Transfer     Bitrate
> [  5]   0.00-10.00  sec  12.0 MBytes  10.1 Mbits/sec                  sender
> [  5]   0.00-10.00  sec  12.0 MBytes  10.1 Mbits/sec                  receiver
> 
> iperf Done.
> 
> Session on B:
> 
> root@:~ # netstat -w 1 -I vmx0
>             input           vmx0           output
>    packets  errs idrops      bytes    packets  errs      bytes colls
>          0     0     0          0          0     0          0     0
>          0     0     0          0          0     0          0     0
>        649     0     0     960562        330     0      21800     0
>        819     0     0    1233702        415     0      27390     0
>        910     0     0    1370780        459     0      30294     0
>        819     0     0    1233702        415     0      27390     0
>        910     0     0    1370780        459     0      30294     0
>        910     0     0    1370780        460     0      30360     0
>        819     0     0    1233702        414     0      27324     0
>        910     0     0    1370780        460     0      30360     0
>        819     0     0    1233702        414     0      27324     0
>        910     0     0    1370780        460     0      30360     0
>        285     0     0     412287        147     0       9981     0
>          0     0     0          0          0     0          0     0
>          0     0     0          0          0     0          0     0
>          0     0     0          0          0     0          0     0
> 
> 
> The ACK packets replied from B (the FreeBSD VM) are too aggressive. They are
> about one half of TCP packets received from A.
> 
> I've tested with different bitrates, from 10m to 300m, all behave the same.
> Tested with baremetal FreeBSD 13.1 Box as B (with intel em driver), the 
> bitrates is 1g, also  behaves the same.
> 
> Also tried different FreeBSD versions, 11.4, 12.3, stable/13 and current/14 all 
> behave the same.
> 
> 
> My question is, is that the expected behavior of current default TCP stack?
That is what I would expect. TCP (on FreeBSD) is acking every other packet. This
is also what is specified. MacOS, at least newer versions, send less ACKs.

Best regards
Michael
> 
> 
> 
> Best regards,
> Zhenlei
>