thread scheduling at mutex unlock

[Andriy Gapon]

on 15/05/2008 15:57 David Xu said the following:
> Andriy Gapon wrote:
>>
>> Maybe. But that's not what I see with my small example program. One 
>> thread releases and re-acquires a mutex 10 times in a row while the 
>> other doesn't get it a single time.
>> I think that there is a very slim chance of a blocked thread 
>> preempting a running thread in this circumstances. Especially if 
>> execution time between unlock and re-lock is very small.
> It does not depends on how many times your thread acquires or 
> re-acquires mutex,  or
> how small the region the mutex is protecting. as long as current thread 
> runs too long,
> other threads will have higher priorities and the ownership definitely 
> will be transfered,
> though there will be some extra context switchings.

David,

did you examine or try the small program that I sent before?
The "lucky" thread slept for 1 second each time it held mutex.
So in total it spent about 8 seconds sleeping and holding the mutex. And 
the "unlucky" thread, consequently, spent 8 seconds blocked waiting for 
that mutex. And it didn't get "lucky".
Yes, technically the "lucky" thread was not running while holding the 
mutex, so probably this is why scheduling algorithm didn't immediately work.

I did more testing and see that the "unlucky" thread eventually gets a 
chance (eventually means after very many lock/unlock cycles), but I 
think that it is penalized too much still.
I wonder if with current code it is possible and easy to make this 
behavior more deterministic.
Maybe something like the following:
if (oldest_waiter.wait_time < X)
    do what we do now...
else
    go into kernel for possible switch

I have very little idea about unit and value of X.

>> I'd rather prefer to have an option to have FIFO fairness in mutex 
>> lock rather than always avoiding context switch at all costs and 
>> depending on scheduler to eventually do priority magic.
>>
> It is better to implement this behavior in your application code,  if it
> is implemented in thread library, you still can not control how many
> times acquiring and re-acquiring  can be allowed  for a thread without
> context switching, a simple FIFO as you said here will cause dreadful
> performance problem.

I almost agree. But I still wouldn't take your last statement for a 
fact. "Dreadful performance"  - on micro-scale maybe, not necessarily on 
macro scale.
After all, never switching context would be the best performance for a 
single CPU-bound task, but you wouldn't think that this is the best 
performance for the whole system.

As a data point: it seems that current Linux threading library is not 
significantly worse than libthr, but my small test program on Fedora 7 
works to my expectations.

-- 
Andriy Gapon