ule+smp: small optimization for turnstile priority lending

[Andriy Gapon]

on 20/09/2012 14:04 Andriy Gapon said the following:
> on 19/09/2012 10:33 Attilio Rao said the following:
>> It is hard for me to tell if this is subject to same issues because I
>> do not entirely understand where the locking was happening in your
>> patch.
>> Can you try testing this with your already KTR instrumented test and
>> possibly report?
> 
> The patch works well as far as I can tell.  Thank you!

After more testing it seems that the idea was not complete.  While loads are set
better tdq_lowpri (which is generally <= newpri) on the current CPU still
prevents the boosted thread from taking advantage of the lowered load.
Not sure how to "refresh" tdq_lowpri in this scenario...

> There is one warning with full witness enables but it appears to be harmless (so
> far):
> 
> Acquiring duplicate lock of same type: "sched lock"
>  1st sched lock 1 @ /usr/src/sys/kern/subr_turnstile.c:212
>  2nd sched lock 0 @ /usr/src/sys/kern/sched_ule.c:1244
> KDB: stack backtrace:
> db_trace_self_wrapper() at 0xffffffff802d238a = db_trace_self_wrapper+0x2a
> kdb_backtrace() at 0xffffffff80555f4a = kdb_backtrace+0x3a
> _witness_debugger() at 0xffffffff8056e2bc = _witness_debugger+0x2c
> witness_checkorder() at 0xffffffff8056f759 = witness_checkorder+0x579
> _mtx_lock_spin_flags() at 0xffffffff80504bcd = _mtx_lock_spin_flags+0x10d
> sched_pickcpu() at 0xffffffff80547829 = sched_pickcpu+0x199
> sched_add() at 0xffffffff80547bdb = sched_add+0x14b
> sched_thread_priority() at 0xffffffff80548059 = sched_thread_priority+0x1c9
> sched_lend_prio() at 0xffffffff80548344 = sched_lend_prio+0x14
> propagate_priority() at 0xffffffff8056801e = propagate_priority+0x1ce
> turnstile_wait() at 0xffffffff8056959f = turnstile_wait+0x4ef
> _mtx_lock_sleep() at 0xffffffff805045f6 = _mtx_lock_sleep+0x486
> _mtx_lock_flags() at 0xffffffff80504814 = _mtx_lock_flags+0x104
> lock_mtx() at 0xffffffff805049ca = lock_mtx+0x1a
> _sleep() at 0xffffffff80524589 = _sleep+0x4f9
> taskqueue_thread_loop() at 0xffffffff805664c8 = taskqueue_thread_loop+0xa8
> fork_exit() at 0xffffffff804e5d6a = fork_exit+0x1aa
> fork_trampoline() at 0xffffffff806ea2ce = fork_trampoline+0xe

-- 
Andriy Gapon