Re: Avoiding the lagg(4) bottleneck

All:

The LAG configuration is about as simple as it can be:

ifconfig_igb2="up -tso4 -tso6 -lro -vlanhwtso media 1000baseTX 
mediaopt full-duplex"
ifconfig_igb3="up -tso4 -tso6 -lro -vlanhwtso media 1000baseTX 
mediaopt full-duplex"
cloned_interfaces="lagg0"
ifconfig_lagg0="laggproto lacp laggport igb2 laggport igb3 lacp_fast_timeout"

Other sysctls of possible interest (from /boot/loader.conf):

# Disable hyperthreading
machdep.hyperthreading_allowed="0"
# Allowing multithreaded networking yields major load reduction
net.isr.maxthreads="-1"
# Optimizations for NICs
# This is claimed to provide improvement with em/igb driver
hw.em.rx_process_limit="-1"
# Don't be overwhelmed by interrupts
hw.em.max_interrupt_rate=10000
# Don't try to save energy on em/igb interface
hw.em.eee_setting=0

We have tried

net.link.lagg.default_use_flowid=0

but it does not seem to matter.

Our concern is that the lagg(4) driver may be limited by single 
threading and/or locks.
We are testing with real life loads, attempting to saturate a pair of gigabit
upstream connections. Try as we might, we can't get 2 Gbps out of 
it; in fact, we
have trouble getting 1 Gbps out of the pair.

--Brett Glass

At 10:05 PM 4/20/2025, Paul Procacci wrote:

>What is your lagg configuration?
>How many different ip/port source tuples are involved with your testing?
>How are you testing?
>
>There's no known deficiency with FreeBSD's lagg driver.
>
>~Paul
>
>--
>__________________
>
>:(){ :|:& };: