[Bug 233759] igb (I210) + net.inet.ipsec.async_crypto=1 + aesni kills receiving queues and traffic

[bugzilla-noreply at freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=233759

            Bug ID: 233759
           Summary: igb (I210) + net.inet.ipsec.async_crypto=1 + aesni
                    kills receiving queues and traffic
           Product: Base System
           Version: CURRENT
          Hardware: Any
                OS: Any
            Status: New
          Severity: Affects Only Me
          Priority: ---
         Component: kern
          Assignee: bugs at FreeBSD.org
          Reporter: lev at FreeBSD.org

I'm benchmarking system with FreeBSD 13-CURRENT (r340933) configured as "VPN
border router". It receives traffic via igb0 (I210) and pass it with encryption
via IPsec tunnel mode via igb1 (I210 too).

CPU supports AES-NI, so it is enabled. I'm using aes-128-gcm cipher.

Incoming traffic has characteristics which allow to employ all 4 queues of
igb0, which run on 4 cores of 4-core CPU, it could be seen in "top -SH" output
and under heavy load queues consume 400% of CPU, 100% each. Everything works as
expected.

But when I set "sysctl net.inet.ipsec.async_crypto=1" strange thing happens:
some igb0 queues stop to work. Sometimes it is 1 queue, sometimes 2 queues, and
sometimes 3 (I never seen 4 queues stop to work, though). Stopped queues don't
consume CPU and don't pass traffic, so 1/4, 2/4 or 3/4 of incoming traffic is
simply lost. Even if traffic is very light (say, 64 packet per second!),
traffic which was classified to "stopped" queues is lost. "tcpdump -i igb0"
doesn't show it.

Only reboot reset this behavior, setting "sysctl net.inet.ipsec.async_crypto=0"
doesn't help.

-- 
You are receiving this mail because:
You are the assignee for the bug.