Avoiding the lagg(4) bottleneck

From: Brett Glass <brett_at_lariat.net>
Date: Mon, 21 Apr 2025 01:48:09 UTC
All:

I've been attempting to expand the capacity of the connection 
between a FreeBSD machine and a switch by using the lagg(4) device 
to aggregate two gigabit ports.

Unfortunately, I'm running into a bottleneck. Perhaps due to the 
overhead of the driver, or limits on multithreading, or locks 
within the system, I can't get anywhere near 2 Gbps out of the pair 
of gigabit ports.

The system CPU is barely loaded and has plenty of compute power, 
and each port can saturate when used alone.

Where's the bottleneck, and is there a way to get around it? 
Perhaps using Netgraph rather than lagg(4)? Ideas welcome.

--Brett Glass