Changes in the network interface queueing handoff model

[Robert Watson]

On Tue, 1 Aug 2006, Andrew Gallatin wrote:

> Robert Watson writes:
>
> >   The immediate practical benefit is clear: if the queueing at the ifnet 
> > layer is unnecessary, it is entirely avoided, skipping enqueue, dequeue, 
> > and four mutex operations.
>
> This is indeed nice, but for TCP I think the benefit would be far greater if 
> somebody would PLEASE, PLEASE, PLEASE implement TSO (aka LSO).
>
> Consider a 1460 byte mss and 64KB of data that is ready to be sent. With the 
> current model, that is 45 separate calls to if_output(), and 45*4 (queuing) 
> + 45 (tx routine) == 225 mutex operations.
>
> Using your model, we're down to 45 mutex operations.
>
> Using TSO, we have 4 + 1 == 5 mutex operations with the old model, and 1 
> with the your model.
>
> This is not even considering all the other overhead involved in 45 transmits 
> vs TSO...

Jack Vogel at Intel has previously talked about having TSO patches for FreeBSD 
to use with if_em, but was running into stability/correctness problems on 7.x. 
I e-mailed him a few minutes ago to ask to take a look at the patches.  Since 
I've not yet seen them, I don't know the details of how they work -- I assume 
that some amount of tweaking is required not just to TCP so that it passes 
down larger segments, but so that larger segments are used in the right ways, 
and in keeping with the facilities offered by the underlying interface, 
especially before a routing decision has been made.

Robert N M Watson
Computer Laboratory
University of Cambridge