lacp lagg port flags do not show correctly resulting in poor traffic distribution/performance

[Andrew Boyer]

On Jul 9, 2012, at 8:38 PM, Adarsh Joshi wrote:

> Hi,
> 
> I am trying to configure lacp lagg interfaces with 2 systems connected back to back as follows:
> 
> Ifconfig lagg0 create
> Ifconfig lagg0 laggproto lacp laggport ql0 laggport ql1 192.168.100.1 netmask 255.255.255.0
> 
> Sometimes, the lag interface comes up correctly but sometimes the laggport flags do not show properly. Instead of 1c<ACTIVE,COLLECTING,DISTRIBUTING>, it shows values of 18. I have seen similar issues reported on various forums with no solution.
> Looking at the lagg driver code and reading the standard, I thought the laggport flags ( defined in if_lagg.h) are based on the LACP_STATE_BITS in file ieee8023ad_lacp.h. But the following ifconfig -v output does not make any sense to me.
> 
> My concern is that when all the interfaces show flags as 1c, the traffic is distributed across both the interfaces uniformly and I get aggregated throughput. If not, the traffic flows only on 1 interface.
> 
> Is this a bug? How do I solve this? Or am I doing something wrong?
> 
> I am using Free-BSD 9.0 release.
> 
> System 1:
> # ifconfig -v lagg0
>        lag id: [(8000,00-0E-1E-08-05-20,0213,0000,0000),
>                 (8000,00-0E-1E-04-2C-F0,0213,0000,0000)]
>        laggport: ql1 flags=18<COLLECTING,DISTRIBUTING> state=7D
>                [(8000,00-0E-1E-08-05-20,0213,8000,000F),
>                 (FFFF,00-00-00-00-00-00,0000,FFFF,0000)]
>        laggport: ql0 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING> state=3D
>                [(8000,00-0E-1E-08-05-20,0213,8000,000E),
>                 (8000,00-0E-1E-04-2C-F0,0213,8000,000E)]
> 
> System 2:
> 
> # ifconfig -v lagg0
>        lag id: [(8000,00-0E-1E-04-2C-F0,0213,0000,0000),
>                 (FFFF,00-00-00-00-00-00,0000,0000,0000)]
>        laggport: ql1 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING> state=7D
>               [(8000,00-0E-1E-04-2C-F0,0213,8000,000F),
>                 (FFFF,00-00-00-00-00-00,0000,FFFF,0000)]
>        laggport: ql0 flags=18<COLLECTING,DISTRIBUTING> state=3D
>                [(8000,00-0E-1E-04-2C-F0,0213,8000,000E),
>                 (8000,00-0E-1E-08-05-20,0213,8000,000E)]
> 
> 
> thanks
> Adarsh
> 


I don't think you have a port flags problem per se; the flags are correctly displaying the state of the lagg.  Your problem is that your systems aren't negotiating the correct lagg configuration.  Each tuple after the laggport represents the [(actor state),(partner state)].  Ports ql0 have been able to talk to their partners (each other).  Neither ql1 port has seen a response from a partner, though.

You could try restarting the state machine on one box with 'ifconfig lagg0 laggproto lacp'.  To see the negotiation you'll need to rebuild your kernel with '#define LACP_DEBUG 1' added to the top of sys/net/ieee802.3ad_lacp.c.  Or upgrade to a newer stable snapshot that has the net.lacp_debug sysctl and turn it on.

Or just turn off LACP.  What does it get you in this configuration?

Hope this helps,
  Andrew

--------------------------------------------------
Andrew Boyer	aboyer at averesystems.com