kern/176097: lagg/lacp broken when aggregated interfaces have same speed but different media type

Wed Feb 13 07:50:01 UTC 2013

>Number:         176097
>Category:       kern
>Synopsis:       lagg/lacp broken when aggregated interfaces have same speed but different media type
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          change-request
>Submitter-Id:   current-users
>Arrival-Date:   Wed Feb 13 07:50:00 UTC 2013
>Closed-Date:
>Last-Modified:
>Originator:     Josef Pojsl
>Release:        8.3-RELEASE-p3
>Organization:
TNS, a.s.
>Environment:
FreeBSD FW-C-02.kr-jihomoravsky.cz 8.3-RELEASE-p3 FreeBSD 8.3-RELEASE-p3 #24: Tue Jun 12 23:26:38 CEST 2012     root at builder864.pha.tns.cz:/usr/obj/usr/src/sys/KERNUN.amd64  amd64
>Description:
I have aggregated two interfaces with the same speed but slightly different type of media (namely 10Gbase-SR and 10Gbase-LR). There is a Cisco switch on the other side.

LACP won't work as my FreeBSD box computes the actor key differently for the two interfaces. This is weird as LACP inists on the same speed but not on the exact same type of media. Cisco has no problem having one aggregated interface Short and the other Long Range.

Outputs of ifconfig:
[root at FW-C-02 /var/log]# ifconfig ix0
ix0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=401bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWTSO>
	ether 90:e2:ba:26:3a:18
	media: Ethernet autoselect (10Gbase-SR <full-duplex>)
	status: active
[root at FW-C-02 /var/log]# ifconfig ix1
ix1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=401bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWTSO>
	ether 90:e2:ba:26:3a:18
	media: Ethernet autoselect (10Gbase-LR <full-duplex>)
	status: active
[root at FW-C-02 /var/log]# ifconfig -v lagg0
lagg0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: LAGG  options=401bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWTSO>
        ether 90:e2:ba:26:3a:18
        media: Ethernet autoselect
        status: active
        groups: lagg 
        laggproto lacp
        lag id: [(8000,90-E2-BA-26-3A-18,0353,0000,0000),
                 (8000,20-FD-F1-DF-9A-84,000A,0000,0000)]
        laggport: ix1 flags=0<> state=D
                [(8000,90-E2-BA-26-3A-18,0352,8000,0002),
                 (8000,20-FD-F1-DF-9A-84,000A,8000,00B3)]
        laggport: ix0 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING> state=3D
                [(8000,90-E2-BA-26-3A-18,0353,8000,0001),
                 (8000,20-FD-F1-DF-9A-84,000A,8000,00B2)]

Please note that the actor key is 0352 (for ix1) and 0353 (for ix0) respectively.
>How-To-Repeat:
Aggregate two ix interfaces, one short-range, the other one long-range, into a lagg interface using lacp.

Excerpt from rc.conf:
ifconfig_ix0=" up descr OPT10SR"
ifconfig_ix1=" up descr OPT10LR"
ifconfig_lagg0="inet 127.0.1.1 netmask 255.255.255.255 descr LAGG laggport ix0 laggport ix1 laggproto lacp"
>Fix:
I have found the place in code where actor key is computed.  When using the same key for media types of the same speed, lacp starts working. Please see the attached patch file.

The ok result can be demonstrated with ifconfig:

[root at FW-C-02 /var/log]# ifconfig -v lagg0
lagg0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
	description: LAGG
	options=401bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWTSO>
	ether 90:e2:ba:26:3a:18
	inet 127.0.1.1 netmask 0xffffffff broadcast 127.0.1.1
	media: Ethernet autoselect
	status: active
	groups: lagg 
	laggproto lacp
	lag id: [(8000,90-E2-BA-26-3A-18,0352,0000,0000),
		 (8000,20-FD-F1-DF-9A-84,000A,0000,0000)]
	laggport: ix1 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING> state=3D
		[(8000,90-E2-BA-26-3A-18,0352,8000,0002),
		 (8000,20-FD-F1-DF-9A-84,000A,8000,00B3)]
	laggport: ix0 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING> state=3D
		[(8000,90-E2-BA-26-3A-18,0352,8000,0001),
		 (8000,20-FD-F1-DF-9A-84,000A,8000,00B2)]

Patch attached with submission follows:

*** sys/net/ieee8023ad_lacp.c.orig	2013-02-12 21:03:16.000000000 +0100
--- sys/net/ieee8023ad_lacp.c	2013-02-12 21:13:39.000000000 +0100
***************
*** 1028,1035 ****
  		KASSERT(IFM_TYPE(media) == IFM_ETHER, ("invalid media type"));
  		KASSERT((media & IFM_FDX) != 0, ("aggregating HDX interface"));

! 		/* bit 0..4:	IFM_SUBTYPE */
! 		key = subtype;
  		/* bit 5..14:	(some bits of) if_index of lagg device */
  		key |= 0x7fe0 & ((sc->sc_ifp->if_index) << 5);
  		/* bit 15:	0 */
--- 1028,1063 ----
  		KASSERT(IFM_TYPE(media) == IFM_ETHER, ("invalid media type"));
  		KASSERT((media & IFM_FDX) != 0, ("aggregating HDX interface"));

! 		/* bit 0..4:	IFM_SUBTYPE modulo speed */
! 		switch (subtype) {
! 		case IFM_10_T:
! 		case IFM_10_2:
! 		case IFM_10_5:
! 		case IFM_10_STP:
! 		case IFM_10_FL:
! 			key = IFM_10_T; break;
! 		case IFM_100_TX:
! 		case IFM_100_FX:
! 		case IFM_100_T4:
! 		case IFM_100_VG:
! 		case IFM_100_T2:
! 			key = IFM_100_TX; break;
! 		case IFM_1000_SX:
! 		case IFM_1000_LX:
! 		case IFM_1000_CX:
! 		case IFM_1000_T:
! 			key = IFM_1000_SX; break;
! 		case IFM_10G_LR:
! 		case IFM_10G_SR:
! 		case IFM_10G_CX4:
! 		case IFM_10G_TWINAX:
! 		case IFM_10G_TWINAX_LONG:
! 		case IFM_10G_LRM:
! 		case IFM_10G_T:
! 			key = IFM_10G_LR; break;
! 		default:
! 			key = subtype;
! 		}
  		/* bit 5..14:	(some bits of) if_index of lagg device */
  		key |= 0x7fe0 & ((sc->sc_ifp->if_index) << 5);
  		/* bit 15:	0 */

>Release-Note:
>Audit-Trail:
>Unformatted: