thread scheduling at mutex unlock

[Daniel Eischen]

On Fri, 16 May 2008, Brent Casavant wrote:

> So, let's get back to the original situation.  What we have is a
> traffic light that's allowing traffic in one direction 99% of
> the time, and the other direction 1% of the time.  _Of_course_
> the traffic on the 1% direction is going to encounter delays
> and starvation; the traffic control mechanism is being used
> incorrectly!  The solution is not to replace it with a stop sign,
> which admittedly makes everything more "fair", because it comes
> at the incredibly high price of never allowing the free flow of
> traffic in the other direction.

I don't think this is really the problem - from what I understood,
the lock is not held for a long amount of time without being
dropped.  It is just that more events are arriving and the
lock is taken and dropped more often as events (objects, whatever)
are added to the queue.  It isn't a "take lock, process many
events, add each to queue, release lock".

This problem may be made more obtuse in the case of SMP.
Consider two CPUs, with each thread running on a separate
CPU.  When one thread releases the lock, sees that it was
contested, and reenters the kernel to wakeup the other
thread, it immediately returns and beats the other woken
thread to the mutex.  Even a yield() might not help - the
scheduler might want to keep the starving thread on the
other CPU.

I think to be fair, the contested mutex case should try
to handoff the mutex, in lieu of any priority protocol
that is in place for the threads or mutex.  And actually,
I think in order to properly implement priority mutexes,
there must be a handoff.

I think it would be very hard for the scheduler to solve
this problem since it doesn't know the dependencies the
threads have on the mutex (or any other userland resource).

There are other ways to solve this in the application,
something like I posted yesterday, perhaps a priority
ceiling mutex with the starving thread having an elevated
priority.  But unfortunately, I think that only works if
the application is running with superuser privileges (with
libkse it will/would work without su privileges).

-- 
DE