Chelsio NETMAP performance
Slawa Olhovchenkov
slw at zxy.spb.ru
Tue Feb 25 15:57:42 UTC 2020
On Wed, Feb 05, 2020 at 05:59:14PM -0800, Navdeep Parhar wrote:
> On Wed, Feb 05, 2020 at 02:38:32PM +0300, Slawa Olhovchenkov wrote:
> > On Tue, Feb 04, 2020 at 12:37:08PM -0800, Navdeep Parhar wrote:
> >
> > > >> nm_holdoff_tmr_idx is a 0-based index into the list above. So if the
> > > >> tmr idx is 0 you are using the 0th (first) value from the list of
> > > >> timers. Try increasing nm_holdoff_tmr_idx and see if that brings down
> > > >> the interrupt rate under control.
> > > >>
> > > >> # sysctl hw.cxgbe.nm_holdoff_tmr_idx=3/4/5
> > > >
> > > > OK, interrupt rate go down, but interrupt time about same.
> > > > (interrupt rate for intel card about 0, compared to 25% chelsio).
> > >
> > > I think iflib runs a lot of stuff in taskqueues rather than the driver
> > > ithread so the CPU accounting may vary. Use dtrace to see if
> >
> > Don't think this is impact: worker's CPU core w/o any syscalls and
> > only w/ bunding workker thread and NIC irq handler show about 100%
> > user CPU time.
> >
> > May be some cache-miss work performed later, at poll(2) time in case
> > of intel driver compared to chelsio (do at interrupt time)?
>
> Could be. While we are here, is it possible for you to try the patches
> in these two?
>
> https://reviews.freebsd.org/D17868
about ~10% performance drop
> https://reviews.freebsd.org/D17869
about ~3-5% performance rise.
> >
> > > netmap_rx_irq is being called by an ithread or a taskqueue to figure out
> > > what driver does what.
> >
> > Can you explain some more?
> > I am not sure about dtrace probe to use and later evaluation
>
> # dtrace -n 'fbt::netmap_rx_irq:entry {stack()}'
>
> Take a look at the stack and see if it's an ithread or one of iflib's
> taskqueues that called netmap_rx_irq.
>
> Regards,
> Navdeep
More information about the freebsd-net
mailing list