lacp lagg port flags do not show correctly resulting in poor traffic distribution/performance

[Adarsh Joshi]

Andrew,

Thanks for the reply.

The reason for my suspicion on the portflags is thus (extracted from the ifconfig output in my previous mail):

System 1:
Laggport: ql1 flags = 18 state = 7D
Laggport: ql0 flags = 1c state = 3D

System 2:
Laggport: ql1 flags = 1c state = 7D
Laggport: ql0 flags = 18 state = 3D

I should have explained my setup to you before. Here it is.
Both the ql0 interfaces of the 2 systems are connected using a single cable and ql1 interfaces of the 2 systems are connected using a single cable.

               System 1                             System 2
                              ql0 <=======> ql0
                              ql1 <=======> ql1

With this setup, I don't think it is possible for ports ql0 to talk to their partners (each other) and ql1 ports not getting a response from their partner and still get the lagg configuration I have posted.

I thought the portflags are dependent on the LACP state. But I see different flags for the same LACP state (For the state 7D, ql1 on system 1 shows flags = 18 and ql1 on system 2 shows flags = 1c).

Or is my understanding totally wrong?

I will send the LACP_DEBUG logs within the hour.

Thanks
Adarsh

From: Andrew Boyer [mailto:aboyer at averesystems.com]
Sent: Tuesday, July 10, 2012 5:57 AM
To: Adarsh Joshi
Cc: freebsd-net at freebsd.org
Subject: Re: lacp lagg port flags do not show correctly resulting in poor traffic distribution/performance


On Jul 9, 2012, at 8:38 PM, Adarsh Joshi wrote:


Hi,

I am trying to configure lacp lagg interfaces with 2 systems connected back to back as follows:

Ifconfig lagg0 create
Ifconfig lagg0 laggproto lacp laggport ql0 laggport ql1 192.168.100.1 netmask 255.255.255.0

Sometimes, the lag interface comes up correctly but sometimes the laggport flags do not show properly. Instead of 1c<ACTIVE,COLLECTING,DISTRIBUTING>, it shows values of 18. I have seen similar issues reported on various forums with no solution.
Looking at the lagg driver code and reading the standard, I thought the laggport flags ( defined in if_lagg.h) are based on the LACP_STATE_BITS in file ieee8023ad_lacp.h. But the following ifconfig -v output does not make any sense to me.

My concern is that when all the interfaces show flags as 1c, the traffic is distributed across both the interfaces uniformly and I get aggregated throughput. If not, the traffic flows only on 1 interface.

Is this a bug? How do I solve this? Or am I doing something wrong?

I am using Free-BSD 9.0 release.

System 1:
# ifconfig -v lagg0
       lag id: [(8000,00-0E-1E-08-05-20,0213,0000,0000),
                (8000,00-0E-1E-04-2C-F0,0213,0000,0000)]
       laggport: ql1 flags=18<COLLECTING,DISTRIBUTING> state=7D
               [(8000,00-0E-1E-08-05-20,0213,8000,000F),
                (FFFF,00-00-00-00-00-00,0000,FFFF,0000)]
       laggport: ql0 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING> state=3D
               [(8000,00-0E-1E-08-05-20,0213,8000,000E),
                (8000,00-0E-1E-04-2C-F0,0213,8000,000E)]

System 2:

# ifconfig -v lagg0
       lag id: [(8000,00-0E-1E-04-2C-F0,0213,0000,0000),
                (FFFF,00-00-00-00-00-00,0000,0000,0000)]
       laggport: ql1 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING> state=7D
              [(8000,00-0E-1E-04-2C-F0,0213,8000,000F),
                (FFFF,00-00-00-00-00-00,0000,FFFF,0000)]
       laggport: ql0 flags=18<COLLECTING,DISTRIBUTING> state=3D
               [(8000,00-0E-1E-04-2C-F0,0213,8000,000E),
                (8000,00-0E-1E-08-05-20,0213,8000,000E)]


thanks
Adarsh

I don't think you have a port flags problem per se; the flags are correctly displaying the state of the lagg.  Your problem is that your systems aren't negotiating the correct lagg configuration.  Each tuple after the laggport represents the [(actor state),(partner state)].  Ports ql0 have been able to talk to their partners (each other).  Neither ql1 port has seen a response from a partner, though.

You could try restarting the state machine on one box with 'ifconfig lagg0 laggproto lacp'.  To see the negotiation you'll need to rebuild your kernel with '#define LACP_DEBUG 1' added to the top of sys/net/ieee802.3ad_lacp.c.  Or upgrade to a newer stable snapshot that has the net.lacp_debug sysctl and turn it on.

Or just turn off LACP.  What does it get you in this configuration?

Hope this helps,
  Andrew

--------------------------------------------------
Andrew Boyer       aboyer at averesystems.com<mailto:aboyer at averesystems.com>





________________________________
This message and any attached documents contain information from QLogic Corporation or its wholly-owned subsidiaries that may be confidential. If you are not the intended recipient, you may not read, copy, distribute, or use this information. If you have received this transmission in error, please notify the sender immediately by reply e-mail and then delete this message.