Re: Slow WAN traffic to FreeBSD hosts but not to Linux hosts---how to debug/fix?
Date: Wed, 01 Feb 2023 02:46:10 UTC
On 1/31/23 13:38, Marek Zarychta wrote: > W dniu 31.01.2023 o 19:31, Paul Mather pisze: >>> While playing with different mod_cc(4) might bring some improvement, >>> to get a real boost I'd suggest enabling tcp_rack(4) if feasible. >> >> I am interested in trying this out, but believe it is more feasible in >> my case for the -STABLE and -CURRENT systems I am using, not so much >> for the -RELEASE systems that are kept up to date via binary >> freebsd-update updates. My reading of the tcp_rack(4) man page is >> that you have to build a custom kernel as, unlike the cc_* congestion >> control algorithms, the loadable tcp_rack module is not built by >> default. Is that an accurate reading? >> > Yes, this gift from Netflix is probably better suited for -STABLE and > -CURRENT as easier to set up there. There is an excellent, up-to-date > article about it by Klara Systems writers[1]. From my experience > tcp_rack(4) is well suited for congested, lossy or redundant network > paths where loses, duplicated packets or races between packets occur. > Not a panacea, but very performant TCP stack based on the _fair_ > algorithm. In some instances, it might help you to saturate the > bandwidth of the link. TCP algo can be loaded/unloaded/changed on the > fly. In FreeBSD 14-CURRENT you can change it on an active socket with > tcpsso(8) utility, in FreeBSD 12 and 13 you have to restart the app > bound to the socket. > Please feel free to play with TCP stacks and congestion algos with the > help of benchmarks/iperf3 to find out what prevents the link from being > saturated and give us some feedback here. > > [1] https://klarasystems.com/articles/using-the-freebsd-rack-tcp-stack/ > > Cheers > I compiled a custom kernel (releng/13.1) and followed Klara Systems instructions. The results are quite good. I would hope the RACK stack will be included in the upcoming 13.2 release as it is a significant upgrade. Testing: ------------------------------------ All computers are running FreeBSD 13.1. File transfer done with SCP. Route to HOST has 17 hops and 110ms pings. Connection limited to 100mb. Congestion tonight was highly variable, so the data below is very rough. All CC algorithms did better under RACK TCP Stack freebsd (default) ====================================== htcp 100% 315MB 3.7MB/s 01:24 htcp 100% 315MB 3.1MB/s 01:41 htcp 100% 315MB 2.6MB/s 02:03 newreno 100% 315MB 4.8MB/s 01:05 newreno 100% 315MB 1.4MB/s 03:48 newreno 100% 315MB 1.7MB/s 03:01 cubic 100% 315MB 8.3MB/s 00:37 cubic 100% 315MB 1.1MB/s 04:49 cubic 100% 315MB 1.9MB/s 02:44 TCP Stack RACK ====================================== htcp 100% 315MB 3.9MB/s 01:19 htcp 100% 315MB 8.6MB/s 00:36 htcp 100% 315MB 10.6MB/s 00:29 newreno 100% 315MB 10.5MB/s 00:30 newreno 100% 315MB 9.1MB/s 00:34 newreno 100% 315MB 7.6MB/s 00:41 cubic 100% 315MB 4.1MB/s 01:16 cubic 100% 315MB 8.6MB/s 00:36 cubic 100% 315MB 7.3MB/s 00:42 -- David