How to Increase TX Queue Priority for LACP Packets

[Navdeep Parhar]

We could have a global knob that tells all NIC drivers to use a reserved
queue for non-RSS traffic, but that would be advisory at best because
the tx queue selection takes place inside the driver's (or iflib's)
transmit routine.  The meat of the change is going to be in iflib and
all non-iflib drivers' if_transmit.

Regards,
Navdeep

On Tue, Jun 16, 2020 at 09:48:19PM +0000, Saad, Mark wrote:
> All
>  Is there any way to make this change on other nic's like Intel ix and
>  Solarflare sfxge ? I have seen similar issues on both with 12.1
>  mainly with solarflare nics.
> 
> ---
> Mark Saad
> mark.saad at lucera.com
> 
> 
> ________________________________________
> From: owner-freebsd-net at freebsd.org <owner-freebsd-net at freebsd.org> on behalf of Foster, Greg <gfoster at panasas.com>
> Sent: Tuesday, June 16, 2020 3:56 PM
> To: Navdeep Parhar
> Cc: freebsd-net at freebsd.org
> Subject: RE: How to Increase TX Queue Priority for LACP Packets
> 
> HI Navdeep,
> 
> Thanks for the information!  I've integrated the changes and will be
> testing more today.
> 
> We have seen the LACP port flapping under different scenarios, most we
> believe are traffic/load based.
> 
> I did see the flapping unexpectedly when I just enabled LACP debug
> (e.g., sysctl net.link.lagg.lacp.debug=1).     Is this a known
> problem?
> 
> Thanks
> Greg
> 
> -----Original Message-----
> From: Navdeep Parhar <nparhar at gmail.com> On Behalf Of Navdeep Parhar
> Sent: Friday, June 12, 2020 7:51 PM
> To: Foster, Greg <gfoster at panasas.com>
> Cc: freebsd-net at freebsd.org
> Subject: Re: How to Increase TX Queue Priority for LACP Packets
> 
> On Fri, Jun 12, 2020 at 11:47:41PM +0000, Foster, Greg wrote:
> > FreeBSD Networkers,
> >
> > We are seeing LACP port flapping on our FreeBSD 10.4/12.1 systems
> > under different conditions.
> >
> > Can someone explain or point me to the information on how to queue
> > the LACP packets to a higher priority queue ?
> >
> > We are using the Chelsio T580-LP-CR adapter/cxgbe driver.  The
> > Cheslio NICs have 8 TX/RX queues each, but I don't know how to
> > explicitly put the LACP packets in the higher priority TX queue.
> >
> > I've read about PF/ALTQ and think this may be overkill our needs,
> > and was wondering if there was a simpler method.
> 
> This is cxgbe specific but that's what you're using so it'll do.
> 
> Add "hw.cxgbe.rsrv_noflowq=1" to your /boot/loader.conf.  That
> reserves one tx queue for non-RSS traffic (like ARP, LACP).  You might
> also want to increase the number of tx queues to compensate for the
> one that's now reserved.  Use "hw.cxgbe.ntxq=9" for that.  The ntxq
> knob might be different on 10.4 but the man page matching the driver
> should have its exact name.
> 
> Regards,
> Navdeep
> 
> _______________________________________________
> freebsd-net at freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
>