Re: Slow WAN traffic to FreeBSD hosts but not to Linux hosts---how to debug/fix?

On 1/30/23 16:30, Matt Garber wrote:
> 
>      > Any help/insight is gratefully appreciated.
>      >
>      > Cheers,
>      >
>      > Paul.
>      >
> 
>     sysctl net.inet.tcp.cc.algorithm=htcp
> 
>     I would set "htcp" on the server and home computer to improve
>     through in
>     your type of situation.
> 
> 
> 
> There may be other FreeBSD sysctls that have bad defaults in this 
> scenario and could be better tuned, but I doubt changing the CC 
> algorithm at this point is the problem — at least not so much a problem 
> that’s causing throughput to be reduced so drastically. Happy to be 
> wrong if that does help things quickly and easily, though.
> 
> (Since OP mentioned that FreeBSD CC was set to CUBIC, that would match 
> what the Linux boxes are using by default, too, unless they’ve been 
> changed to something newer like BBR… so seems like CUBIC *should* be 
> performing fine on this WAN link, and the difference is something else.)
> 
> 
> —Matt
> 

I love FreeBSD and very much appreciate the efforts of those people much 
smarter. But... I don't think the defaults get enough testing in real 
world conditions.

I came across Paul's issue several years ago and spent a few a hours 
testing and found the defaults performed very well on a LAN but could 
perform terribly on a many hop WAN. HTCP performs marginally worse on a 
LAN or close WAN connection, but much much better on a many hop WAN 
connection.

Testing:
------------------------------------
All computers are running FreeBSD 13.1. File transfer done with SCP. No 
changes were made between runs other than toggling the cc.algorithm. Be 
sure to run sysctl net.inet.tcp.hostcache.expire=1 before testing. 
(default 3600)

HOST 1) 10 hops and 50ms pings. Connection limited to 120mb
------------------------------------
Run #1
NEWRENO 100%  315MB  13.0MB/s   00:24
HTCP    100%  315MB  12.8MB/s   00:24

Run #2
NEWRENO 100% 315MB  13.2MB/s   00:23
HTCP    100%  315MB  12.8MB/s   00:24


HOST 2) 17 hops and 110ms pings. Connection limited to 100mb
------------------------------------
Run #1
NEWRENO  100%  315MB   1.1MB/s   04:48
HTCP     100%  315MB  10.5MB/s   00:30

Run #2
NEWRENO  100%  315MB   1.6MB/s   03:22
HTCP     100%  315MB   3.5MB/s   01:29

Run #3
NEWRENO  100%  315MB   1.3MB/s   04:00
HTCP     100%  315MB  10.8MB/s   00:29

In my opinion HTCP is a better default for the current state of the 
internet.

Cheers,
David