kern/185967: Link Aggregation LAGG: LACP not working in 10.0

[Scott Long]

On Feb 5, 2014, at 1:14 PM, Joe Moog <joemoog at ebureau.com> wrote:

>> Date: Mon, 03 Feb 2014 09:40:30 +0100
>> From: Ben <mailinglists at niessen.ch>
>> To: freebsd-net at freebsd.org
>> Subject: Re: kern/185967: Link Aggregation LAGG: LACP not working in
>> 	10.0
>> Message-ID: <52EF55FE.8030901 at niessen.ch>
>> Content-Type: text/plain; charset=windows-1252; format=flowed
>> 
>> Hi Scott,
>> 
>> I had tried to set it in /etc/sysctl.conf but seems it didnt work. But 
>> will I try again and report back.
>> 
>> The settings of the switch have not been changed and are set to LACP. It 
>> worked before so I guess the switch should not be the problem. Maybe 
>> some incompatibility between FreeBSD + igb-driver + switch (Juniper 
>> EX3300-48T).
>> 
>> I will update you after setting the sysctl setting. It seems to be 
>> "dynamic", I guess 0 reflects the index of LACP lagg devices. Can I 
>> switch off the strict mode globally in /etc/sysctl.conf?
>> 
>> Thanks for your help.
>> 
>> Regards
>> Ben
>> 
>> On 03.02.2014 09:31, Scott Long wrote:
>>> Hi,
>>> 
>>> You?re probably running into the consequences of r253687.  Check to see the value of ?sysctl net.link.lagg.0.lacp.lacp_strict_mode?.  If it?s ?1? then set it to 0.  My original intention was for this to default to 0, but apparently that didn?t happen.  However, the fact that strict mode doesn?t seem to work at all for you might hint that your switch either isn?t configured correctly for LACP, or doesn?t actually support LACP at all.  You might want to investigate that.
>>> 
>>> Scott
>>> 
>>> On Feb 3, 2014, at 1:17 AM, Ben <mailinglists at niessen.ch> wrote:
>>> 
>>>> Hi,
>>>> 
>>>> I upgraded from FreeBSD 9.2-RELEASE to 10.0-RELEASE. FreeBSD 9.2 was configured to use LACP with two igb devices.
>>>> 
>>>> Now it stopped working after the upgrade.
>>>> 
>>>> This is a screenshot of ifconfig -a after the upgrade to FreeBSD 10.0-RELEASE: http://tinypic.com/view.php?pic=28jvgpw&s=5#.Uu9PXT1dVPM
>>>> 
>>>> A PR is currently open: http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/185967
>>>> 
>>>> It is set to low, but I would like somebody to have a look into it as it obviously has a great influence on our infrastructure. The only way to "solve" it is currently switching back to FreeBSD 9.2.
>>>> 
>>>> The suggested fix "use failover" seems not to work.
>>>> 
>>>> Thank you for your help.
>>>> 
>>>> Best regards
>>>> Ben
>>>> 
> 
> Our experience appears to differ. We have 4-pot LAGG configured on an Intel ethernet NIC (igb drivers), connected via LACP to 4 ports on a Cisco Cat4948, host initially configured with FreeBSD 9.2-RELEASE and upgraded to 10.0-RELEASE. Following the upgrade, everything works as expected without making any additional adjustments. (We did initially have to increase the mbuf_cluster allowance to get 4-port LAGG working with 9.2, but that may be immaterial to this conversation.)
> 
> As an outsider looking in, the issue seems to crop up in cases where switch configurations have not been set specifically to force (active) LACP, or it's something related to use with mixed ethernet drivers (e.g., bge mixed with igb, as in the case of the linked PR), or possibly with different switch manufacturer's handling of FreeBSD's LACP negotiation (in both this case and the PR, Juniper). Whether or not this needs to be addressed from within FreeBSD itself I will leave to the experts.


As a follow-up, Ben’s problem was that his switch was set for passive mode and thus not sending out heartbeats.  The FreeBSD LACP driver accidentally switched its default from permissive to strict mode, and in doing so required the reception of heartbeats in order to operate.  Compounding the problem was that the sysctl to change the behavior was completely untested and useless because it set the state of the ports too late in the initialization process.  These problems will be addressed for the 10.1 release.  However, once Ben set his switch to active LACP mode, everything worked.

Scott