Massive performance loss from OS::sleep hack

[Kurt Miller]

Daniel Eischen wrote:
> On Tue, 18 Sep 2007, Kurt Miller wrote:
> 
>> Hi Daniel,
>>
>> Daniel Eischen wrote:
>>> On Tue, 18 Sep 2007, Kurt Miller wrote:
>>>
>>>> David Xu confirmed for me that pthread_yield() does give some
>>>> time to lower priority threads on 7.0 using thr. Attached and inline
>>>> are two patches for the 1.5 port that is how I suggest the issue be
>>>> addressed.
>>>
>>> I don't think you should rely on pthread_yield() doing that,
>>> it is dependent on the scheduler, and if you are running
>>> in SCHED_FIFO/SCHED_RR, it won't work with libthr either.
>>> Currently, you can only run in SCHED_FIFO or SCHED_RR as
>>> root using libthr, but that hopefully won't always be
>>> true.
>>
>> Shoot I forgot to mention a few things in my last email...
>> The c test program I resurrected from 18 months ago had
>>
>> pthread_attr_setschedpolicy(&attr, SCHED_FIFO);
>>
>> in it. The jdk doesn't set the schedpolicy it uses the
>> default one. That line was left over from my testing
>> different policies under kse to see if I could work-
>> around the problem. Since the jdk doesn't set the policy,
>> I believe we don't need to be concerned with SCHED_FIFO
>> and SCHED_RR.
> 
> libkse and libc_r default to SCHED_RR (SCHED_OTHER=SCHED_RR).
> 
>>> I think the way that will always work is to lower the
>>> priority and raise it back again after the yield, when
>>> using thread priorities.  This should work for libthr,
>>> libc_r, and libkse under all versions of the OS.
>>
>> On the surface this sounded promising, but I can envision
>> at least one case where temporarily lowering a thread
>> priority will not work very well. Take three threads A, B &
>> C each with different priorities: A highest, C lowest. Let's
>> say A is in a busy loop calling Thread.Yield(). B is in a busy
>> loop also calling Thread.Yield(). C is doing something that
>> needs some time slices every so often. A calls Thread.Yield(),
>> lowers priority to C's level, B gets some time next, it calls
>> Thread.Yield(), lowers priority to C's level. A, B, C are all
>> at equal level now. Perhaps C will get some time first, or B,
>> or A. Suppose the order is C first, B next.

I forgot to mention here B's priority gets restored until the
next time it calls yield again, creating the possibility of
starvation of A.

>> A will only get
>> a time slice if B continues to call yield and by chance the
>> system schedules A before B. The behavior becomes
>> non-deterministic. The worst case is A is starved.
> 
> No, if the threads are at equal priority, they will always
> run.  The system, either the userland scheduler or the kernel,
> will ensure the threads get scheduled equally.  For the
> kernel scheduler, "equally" depends on their resource usage.
> 
> I would just totally ignore setting thread priorities
> unless the UseThreadPriority knob is set.  The kernel
> scheduler (for libthr) doesn't seem to care what a thread's
> priority is anyways unless it is in the real-time class.
> That way, all threads will be at the default priority
> by default ;-)

I think that's a fine idea. Just changing the default to
be UseThreadPriority=false and completely remove the
os_sleep() bits. If Sun corrects the API or the TCK tests
the default can be changed back.

-Kurt