FYI: inpcb/pcbinfo mutex -> rwlock at some point in the mid-distant future

[Robert Watson]

FYI:

Kris Kennaway and I have been working on diagnosing and addressing network 
stack contention and scheduling issues for high performance DNS serving with 
nsd and bind9.  One of the changes in the pipeline is converting from using a 
mutex to protect the pcbinfo and inpcb data structures to an rwlock, so that 
UDP, which doesn't require exclusion for either of those structures in a 
number of key cases, doesn't have to experience as high contention.  TCP would 
continue to use them only exclusively [for now].  This is a heads up that this 
change will likely end up in HEAD in the next month or two, but we're still 
working through some issues.  This may well be something that can be MFC'd as 
rwlocks are intentionally fairly layout compatible with mutexes, and these 
locks and the data structures they protect are not considered "public" 
structures from the perspective of most kernel modules (although there are 
potentially issues with firewalls that reach "up" the stack to look at th 
connection table).

One of those issues is that we need to demonstrate to ourselves that exclusive 
access contention is managed as well with rwlocks as with sleep mutexes, as 
these locks would continue to be fairly highly contended in TCP.  The other 
issue is that rwlocks don't support full priority propagation for reader 
access, although Jeff Roberson has recently improved fairness to writers with 
many readers.

Robert N M Watson
Computer Laboratory
University of Cambridge