cvs commit: src/sys/dev/mxge if_mxge.c

[Andrew Gallatin]

Gleb Smirnoff writes:
 > On Fri, May 25, 2007 at 10:57:54PM +0200, Andre Oppermann wrote:
 > A>  Andrew Gallatin wrote:
 > A> > Andrew Gallatin [gallatin at FreeBSD.org] wrote:
 > A> >> gallatin    2007-05-25 19:38:32 UTC
 > A> >>
 > A> >>   FreeBSD src repository
 > A> >>
 > A> >>   Modified files:
 > A> >>     sys/dev/mxge         if_mxge.c   Log:
 > A> >>   - Use m_getcl() rather than m_getjcl() when we're allocating 2KB
 > A> >>   clusters.  This helps quite a bit on my low end machines (improves
 > A> >>   performance by about 300Kpps when being blasted by a hardware
 > A> >>   packet generator).
 > A> > BTW, thanks to bmilekic for reminding me of the packet zone.
 > A> > It would be nice if such a thing existed for the jumbo sized
 > A> > clusters :)
 > A> 
 > A>  Secondary zones are not perfectly integrated with UMA and there
 > A>  are some nits that prevent doing just that for the moment.  We're
 > A>  working on it.  It's not trivial though.  I want to de-inline some
 > A>  functions in mbuf.h to make backend changes transparent to consumers
 > A>  of it.
 > 
 > Just 1 penny: Andrews commit shows that packet zone shouldn't be deleted,
 > it really improves performance.

It is not clear that it always improves performance. I think it may
have at least a little to do with the platform (athlon64 rather than
xeon), and the fact that I allocate my receive buffers in the
interrupt thread, rather than having a separate thread to allocate
them.  Both scottl and kmacy have reported substantial gains from
allocating clusters well in advance (in a taskq for cxbe, not sure
what scottl did for em) and allocating the pkthdr mbufs after the
interrupt comes in.  I did not notice an improvement when I tried
allocating pkhdrs late (and I never tried a taskq/thread to allocate
buffers). Again, it might be the platform.  I'll try more experiments
when I get access to Xeons again..

Speaking of mbufs, Bosko pointed out that in revision 1.156 of
kern/uipc_mbuf.c, code was added which clears M_EXT information
at mb_free_ext() time.  He was concerned that doing this at free
time might be cache unfriendly, and that this clearing used
to be done at ctor time.

Drew