Network Stack Locking

[Robert Watson]

On Mon, 24 May 2004, Eivind Eklund wrote:

> On Fri, May 21, 2004 at 01:23:51PM -0400, Robert Watson wrote:
> > The other concern I have is whether the message queues get deep or not: 
> > many of the benefits of message queues come when the queues allow
> > coallescing of context switches to process multiple packets.  If you're
> > paying a context switch per packet passing through the stack each time you
> > cross a boundary, there's a non-trivial operational cost to that.
> 
> I don't know what Matt has done here, but at least with the design we
> used for G2 (a private DFly-like project that John Dyson, I, and a few
> other people I don't know if want to be anonymous or not ran), this
> should not an issue.  We used thread context passing with an API that
> contained putmsg_and_terminate() and message ports that automatically
> could spawn new handler threads.  Effectively, a message-related context
> switch turned into "assemble everything I care about in a small package,
> reset the stack pointer, and go".  The expectation was that this should
> end up with less overhead than function calls, as we could drop the call
> frames for "higher levels in the chain".  We never got to the point
> where we could measure if it worked out that way in practice, though. 

Sounds a lot like a lot of the Mach IPC optimizations, including their use
of continuations during IPC to avoid a full context switch.

Robert N M Watson             FreeBSD Core Team, TrustedBSD Projects
robert at fledge.watson.org      Senior Research Scientist, McAfee Research