rwlocks, correctness over speed.

[Robert Watson]

On Fri, 23 Nov 2007, Stephan Uphoff wrote:

>>> I talked with Attilio about that on IRC. Most common cases of writer 
>>> starvation (but not all) could be solved by keeping a per thread count of 
>>> shared acquired rwlocks. If a rwlock is currently locked as shared/read 
>>> AND a thread is blocked on it to lock it exclusively/write - then new 
>>> shared/read locks will only be granted to thread that already has a shared 
>>> lock. (per thread shared counter is non zero)
>>> 
>>> To be honest I am a bit twitchy about a lock without priority propagation 
>>> - especially since in FreeBSD threads run with user priority in kernel 
>>> space and can get preempted.
>> 
>> That's an interesting hack, I guess it could be done.
>> 
>> I would still like to disallow recursion.
>> 
> Oh - I am all for disallowing recursion. In my opinion the only valid place 
> for a thread to acquire the same lock multiple times is inside a transaction 
> system with full deadlock detection. The question is if we can do that this 
> late in the game? Maybe we could make non recursive the default and add a 
> call rw_allow_recurse or rw_init_recurse to allow recursion on a lock if we 
> can't get away with the straight out removal of the option? (Or less 
> desirable - keep the current default and add functions to disallow 
> recursion)

While I'm no great fan of recursion, the reality is that many of our kernel 
subsystems are not yet ready to disallow recursion on locks.  Take a look at 
the cases where we explicitly enable recursive acquisition for mutexes--in 
practice, most network stack mutexes are recursive due to the recursive 
calling in the network stack.  While someday I'd like to think we'll be able 
to eliminate some of that, but it won't be soon since it requires significant 
reworking of very complicated code.  The current model in which recursion is 
explicitly enabled only where still required seems to work pretty well for the 
existing code, although it's hard to say yet in the code I've looked at 
whether read recursion would be required--the situations I have in mind would 
require purely write recursion.  There's one case in the UNIX domain socket 
code where we do a locked test and conditional lock/unlock with an rwlock for 
exclusive locking because recursion isn't currently supported, and that's not 
a usage I'd like to encourage more of.

Robert N M Watson
Computer Laboratory
University of Cambridge