How to Increase TX Queue Priority for LACP Packets

[Foster, Greg]

HI Navdeep,

Thanks for the information!  I've integrated the changes and will be testing more today.

We have seen the LACP port flapping under different scenarios, most we believe are
traffic/load based.

I did see the flapping unexpectedly when I just enabled LACP debug
(e.g., sysctl net.link.lagg.lacp.debug=1).     Is this a known problem?

Thanks
Greg

-----Original Message-----
From: Navdeep Parhar <nparhar at gmail.com> On Behalf Of Navdeep Parhar
Sent: Friday, June 12, 2020 7:51 PM
To: Foster, Greg <gfoster at panasas.com>
Cc: freebsd-net at freebsd.org
Subject: Re: How to Increase TX Queue Priority for LACP Packets

On Fri, Jun 12, 2020 at 11:47:41PM +0000, Foster, Greg wrote:
> FreeBSD Networkers,
>
> We are seeing LACP port flapping on our FreeBSD 10.4/12.1 systems
> under different conditions.
>
> Can someone explain or point me to the information on how to queue the
> LACP packets to a higher priority queue ?
>
> We are using the Chelsio T580-LP-CR adapter/cxgbe driver.  The Cheslio
> NICs have 8 TX/RX queues each, but I don't know how to explicitly put
> the LACP packets in the higher priority TX queue.
>
> I've read about PF/ALTQ and think this may be overkill our needs, and
> was wondering if there was a simpler method.

This is cxgbe specific but that's what you're using so it'll do.

Add "hw.cxgbe.rsrv_noflowq=1" to your /boot/loader.conf.  That reserves one tx queue for non-RSS traffic (like ARP, LACP).  You might also want to increase the number of tx queues to compensate for the one that's now reserved.  Use "hw.cxgbe.ntxq=9" for that.  The ntxq knob might be different on 10.4 but the man page matching the driver should have its exact name.

Regards,
Navdeep