Intel 4-port ethernet adaptor link aggregation issue

[Ryan Stone]

On Thu, Aug 1, 2013 at 2:07 PM, Joe Moog <joemoog at ebureau.com> wrote:

> We have an iXsystems 1U server (E5) with an Intel 4-port ethernet NIC
> installed, model I350-T4 (manufactured May of 2013). We're trying to bind
> the 4 ports on this NIC together into a single lagg port, connected LACP to
> a distribution switch (Cisco 4900-series). We are able to successfully bind
> the 2 on-board ethernet ports to a single lagg, however the NIC is not so
> cooperative. At first we thought we had a bad NIC, but a replacement has
> not fixed the issue. We are thinking there may be a driver limitation with
> these Intel ethernet NICs when attempting to bind more than 2 ports to a
> lagg.
>
> FreeBSD version:
> FreeBSD 9.1-PRERELEASE #0 r244125: Wed Dec 12 11:47:47 CST 2012
>
> rc.conf:
> # LINK AGGREGATION
> ifconfig_igb2="UP"
> ifconfig_igb3="UP"
> ifconfig_igb4="UP"
> ifconfig_igb5="UP"
> cloned_interfaces="lagg0"
> ifconfig_lagg0="laggproto lacp laggport igb2 laggport igb3 laggport igb4
> laggport igb5"
> ifconfig_lagg0="inet 192.168.1.14  netmask 255.255.255.0"
>
> We've confirmed that the lagg module is loaded (clearly, as the pair of
> on-board ethernet ports can be bound successfully). Binding various
> combinations of ports on the NIC yields odd results, as sometimes the first
> one in the list does not negotiate properly, sometimes the last one in the
> list fails negotiation. Adding interfaces to lagg individually versus all
> at the same time does not seem to make any difference. At one point we even
> tried to assign unique and separate IP addresses to the ethernet ports
> individually, and only a couple of the ports would actually come active and
> respond to any sort of network activity. Due to this issue with the number
> of "usable" ports even beyond the link aggregation failure, this is sort of
> what leads us to believe there may be an issue with the drivers for this
> card.
>
> We've searched the 'net/lists fairly extensively, and have seen very few
> instances where people have tried to bind more than 2 ports to a lagg with
> FreeBSD. Again, 2 ports is no problem, so long as we use the on-board
> ports; it's the introduction of the Intel NIC and 2 more ports that has us
> stuck.
>
> Has anybody had any success with such a setup?
>
> Joe
>
> _______________________________________________
> freebsd-net at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
>

Have you tried using only two ports, but both from the NIC?  My suspicion
would be that the problem is in the lagg's handling of more than 2 ports
rather than the driver, especially given that it is the igb driver in all
cases.