Changes in the network interface queueing handoff model

[John Polstra]

> Attached is a patch that maintains the current if_start, but adds 
> if_startmbuf.  If a device driver implements if_startmbuf and the global 
> sysctl net.startmbuf_enabled is set to 1, then the if_startmbuf path in the 
> driver will be used.  Otherwise, if_start is used.  I have modified the if_em 
> driver to implement if_startmbuf also.  If there is no packet backlog in the 
> if_snd queue, it directly places the packet in the transmit descriptor ring. 
> If there is a backlog, it uses the if_snd queue protected by driver mutex, 
> rather than a separate ifq mutex.

I question whether you need a fallback software if_snd queue at all
for modern devices such as the Intel and Broadcom gigabit chips.  The
hardware transmit descriptor rings typically have sizes of the order
of 256 descriptors.  I think if the ring fills up, you could simply
drop the packet with ENOBUFS.  That's what happens if the if_snd queue
fills up, and its maximum size is comparable to the sizes of modern
descriptor rings.  It would simplify things quite a bit to eliminate
the if_snd queue entirely for such devices.

In any case, I'm glad you're looking at making this change.  I think
it's the right thing to do.

John