dummynet dropping too many packets

[Robert Watson]

On Wed, 7 Oct 2009, rihad wrote:

> rihad wrote:
>> I've yet to test how this direct=0 improves extensive dummynet drops.
>
> Ooops... After a couple of minutes, suddenly:
>
> net.inet.ip.intr_queue_drops: 1284
>
> Bumped it up a bit.

Yes, I was going to suggest that moving to deferred dispatch has probably 
simply moved the drops to a new spot, the queue between the ithreads and the 
netisr thread.  In your setup, how many network interfaces are in use, and 
what drivers?

If what's happening is that you're maxing out a CPU then moving to multiple 
netisrs might help if your card supports generating flow IDs, but most 
lower-end cards don't.  I have patches to generate those flow IDs in software 
rather than hardware, but there are some downsides to doing so, not least that 
it takes cache line misses on the packet that generally make up a lot of the 
cost of processing the packet.

My experience with most reasonable cards is that letting them doing the work 
distribution with RSS and use multiple ithreads is a more performant strategy 
than using software work distribution on current systems, though.

Someone has probably asked for this already, but -- could you send a snapshot 
of the top -SH output in the steady state?  Let top run for a few minutes and 
then copy/paste the first 10-20 lines into an e-mail.

Robert N M Watson
Computer Laboratory
University of Cambridge