How to Increase TX Queue Priority for LACP Packets

[Navdeep Parhar]

On Tue, Jun 16, 2020 at 07:56:19PM +0000, Foster, Greg wrote:
> HI Navdeep,
> 
> Thanks for the information!  I've integrated the changes and will be
> testing more today.
> 
> We have seen the LACP port flapping under different scenarios, most we
> believe are traffic/load based.
> 
> I did see the flapping unexpectedly when I just enabled LACP debug
> (e.g., sysctl net.link.lagg.lacp.debug=1).     Is this a known
> problem?

No, I don't think so.  The debug output goes to the console so it is
relatively slow but it shouldn't cause any flaps directly.

Regards,
Navdeep

> 
> Thanks
> Greg
> 
> -----Original Message-----
> From: Navdeep Parhar <nparhar at gmail.com> On Behalf Of Navdeep Parhar
> Sent: Friday, June 12, 2020 7:51 PM
> To: Foster, Greg <gfoster at panasas.com>
> Cc: freebsd-net at freebsd.org
> Subject: Re: How to Increase TX Queue Priority for LACP Packets
> 
> On Fri, Jun 12, 2020 at 11:47:41PM +0000, Foster, Greg wrote:
> > FreeBSD Networkers,
> >
> > We are seeing LACP port flapping on our FreeBSD 10.4/12.1 systems
> > under different conditions.
> >
> > Can someone explain or point me to the information on how to queue
> > the LACP packets to a higher priority queue ?
> >
> > We are using the Chelsio T580-LP-CR adapter/cxgbe driver.  The
> > Cheslio NICs have 8 TX/RX queues each, but I don't know how to
> > explicitly put the LACP packets in the higher priority TX queue.
> >
> > I've read about PF/ALTQ and think this may be overkill our needs,
> > and was wondering if there was a simpler method.
> 
> This is cxgbe specific but that's what you're using so it'll do.
> 
> Add "hw.cxgbe.rsrv_noflowq=1" to your /boot/loader.conf.  That
> reserves one tx queue for non-RSS traffic (like ARP, LACP).  You might
> also want to increase the number of tx queues to compensate for the
> one that's now reserved.  Use "hw.cxgbe.ntxq=9" for that.  The ntxq
> knob might be different on 10.4 but the man page matching the driver
> should have its exact name.
> 
> Regards,
> Navdeep
>