[Bug 268490] [igb] [lagg] [vlan]: Intel i210 performance degraded

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=268490

            Bug ID: 268490
           Summary: [igb] [lagg] [vlan]: Intel i210 performance degraded
           Product: Base System
           Version: 12.4-RELEASE
          Hardware: Any
                OS: Any
            Status: New
          Severity: Affects Only Me
          Priority: ---
         Component: kern
          Assignee: bugs@FreeBSD.org
          Reporter: dd@goboomtown.com

I have a ZFS file server running ctld to provide an iSCSI target to VMware
hosts. It runs on a Supermicro motherboard which has dual Intel i210 ports on
it. The igb ports are configured in a LACP lagg with the switch, and then vlan
is used to create 4 VLAN interfaces on top of that. The system has been rock
solid stable in production for over a year on 12.2-STABLE. Upgrading it to
12.4-RELEASE 2 weeks ago rendered it unusable as an iSCSI target. I've tried
disabling all the HW options to no avail.

The ctld iSCSI target lives on VLAN 8 (172.27.6.135). When I try to connect to
the target from VMware's initiator, VMware basically hangs and I start seeing
interface errors accumulate on this machine. Once I saw this behavior, I
assumed it was a lower-level network issue and not specific to ctld at all. I
then removed the VMware iSCSI configuration and focused on diagnosing the
lower-level network.

Here's a pair of iperf tests performed on a neighboring FreeBSD 12.4 machine on
the same VLAN segment (172.27.6.135) which has an igb interface without
lagg+vlan. The first test is to this machine's lagg0.8 interface
(172.27.6.135), and the second test is to a third machine (172.27.6.129) which
also has an igb interface without lagg+vlan:

$ for ip in 172.27.6.135 172.27.6.129; do iperf -c ${ip}; done
------------------------------------------------------------
Client connecting to 172.27.6.135, TCP port 5001
TCP window size: 32.0 KByte (default)
------------------------------------------------------------
[  1] local 172.27.6.130 port 22350 connected with 172.27.6.135 port 5001
[ ID] Interval       Transfer     Bandwidth
[  1] 0.00-20.54 sec  43.1 KBytes  17.2 Kbits/sec
------------------------------------------------------------
Client connecting to 172.27.6.129, TCP port 5001
TCP window size: 35.0 KByte (default)
------------------------------------------------------------
[  1] local 172.27.6.130 port 11072 connected with 172.27.6.129 port 5001
[ ID] Interval       Transfer     Bandwidth
[  1] 0.00-10.01 sec  1.15 GBytes   984 Mbits/sec

Here's this machine's config from /etc/rc.conf:

ifconfig_igb0="mtu 9000 media 1000baseTX mediaopt full-duplex -rxcsum -rxcsum6
-txcsum -txcsum6 -lro -tso -vlanhwtso -vlanhwtag -vlanhwcsum -vlanhwfilter up"
ifconfig_igb1="mtu 9000 media 1000baseTX mediaopt full-duplex -rxcsum -rxcsum6
-txcsum -txcsum6 -lro -tso -vlanhwtso -vlanhwtag -vlanhwcsum -vlanhwfilter up"
cloned_interfaces="lagg0"
ifconfig_lagg0="up laggproto lacp laggport igb0 laggport igb1
lacp_fast_timeout"
vlans_lagg0="6 7 8 10"
ifconfig_lagg0_6="inet 172.27.6.10/26 mtu 1500"
ifconfig_lagg0_7="inet 172.27.6.123/26 mtu 1500"
ifconfig_lagg0_8="inet 172.27.6.135/28 mtu 9000"
ifconfig_lagg0_10="inet 172.27.6.251/26 mtu 1500"

Here's this machine's relevant ifconfig output:

igb0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9000
        options=800028<VLAN_MTU,JUMBO_MTU>
        ether 00:25:90:d6:e6:72
        media: Ethernet 1000baseT <full-duplex>
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
igb1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9000
        options=800028<VLAN_MTU,JUMBO_MTU>
        ether 00:25:90:d6:e6:72
        hwaddr 00:25:90:d6:e6:73
        media: Ethernet 1000baseT <full-duplex>
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9000
        options=800028<VLAN_MTU,JUMBO_MTU>
        ether 00:25:90:d6:e6:72
        laggproto lacp lagghash l2,l3,l4
        laggport: igb0 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
        laggport: igb1 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
        groups: lagg
        media: Ethernet autoselect
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lagg0.8: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9000
        ether 00:25:90:d6:e6:72
        inet 172.27.6.135 netmask 0xfffffff0 broadcast 172.27.6.143
        groups: vlan
        vlan: 8 vlanpcp: 0 parent interface: lagg0
        media: Ethernet autoselect
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

-- 
You are receiving this mail because:
You are the assignee for the bug.