HEADS UP: TCP CUBIC Broken on 12.0-RELEASE/STABLE

[Matt Garber]

Hi all:

Just a heads-up for those beginning to test or use FreeBSD 12.0-RELEASE or 12/STABLE, the alternate TCP congestion control algorithm CUBIC (cc_cubic) is currently broken and causes complete stalls of certain network traffic (e.g., rsync, git checkouts, etc.), as well as what appears to be erratic behavior even for basic SSH.

I first noticed on a Digital Ocean VM upgraded from 11.2-RELEASE, but was then able to consistently reproduce on a fresh 12.0-RELEASE AWS EC2 instance merely by loading and enabling cc_cubic instead of the default CC, newreno. H-TCP (cc_htcp) was also tested, but the problem is isolated to CUBIC.

After some discussion on Twitter with Colin Percival, Hiren Panchasara, and others (thank you!), it was determined that the culprit was r331567 which hadn’t been reverted prior to 12.0-RELEASE. For those with custom kernels, reverting that revision in 12/STABLE will fix the problem with cc_cubic (sys/netinet/cc/cc.h, sys/netinet/cc/cc_cubic.c, sys/netinet/cc/cc_cubic.h); for those running GENERIC 12.0-RELEASE kernels, the only immediate workaround is to use newreno or cc_htcp in the meantime. Hiren has already reverted in HEAD (r342127), and an Errata Notice will hopefully be published at some point in January after the holidays.


Hope this saves some troubleshooting!

—
Thanks,
-Matt Garber