igb performance/load udp issue

[Luigi Rizzo]

On Wed, Dec 21, 2011 at 08:04:54AM -0700, Clint wrote:
> On Tue, 20 Dec 2011 22:46:46 -0800, Jack Vogel wrote:
> >I was fighting with UDP issues before the latest checkin, so you 
> >should
> >look at THAT version, 2.3.1 in HEAD please.
> >
> >Jack
> >
> 
> I'm actually running the HEAD version now, but my problems still 
> persist.  I did notice that the excessive load is now spread across 
> multiple cores where before it was isolated to one core at a time.  This 
> version also doesn't suffer from the horrible performance I experienced 
> with v2.2.5.

as someone mentioned, it may have to do with the sender
using multiple queues thus causing reordering in the frames,
which then the receiver has to sort (and often, the receive
queue is a list so O(N) complexity per packet.)

Can you run the igb driver with just a single queue ?
In the bootloader, or before loading the module using "kenv", you
should set

	hw.igb.num_queues = 1

This should prevent gratuitous reordering and perhaps fix
your performance.

cheers
luigi