SCHED_ULE preempt_thresh: PRI_MIN_KERN -> PRI_MIN_IDLE

[Andriy Gapon]

I do not have sufficient knowledge of SCHED_ULE, so maybe I shouldn't even talk
about this, but I couldn't help but notice that many (many) users have reported in
the past heavy interactivity problems with SCHED_ULE under high load, especially
I/O-related load.

The universal advice has always been to tune preempt_thresh via sysctl
kern.sched.preempt_thresh=224.  I think that David Xu was the first person that I
saw recommending this.  In all cases users have reported significant improvements.
 I must add that I also have the experience and I do use preempt_thresh=224 to
this day.

Now, I would like to discuss this phenomenon in two veins:
1. Why do we see the interactivity problem with the default setting of
preempt_thresh=PRI_MIN_KERN (provided that PREEMPTION is enabled and
FULL_PREEMPTION is not)?  Could this be a general ULE issue?  Or could it be
because of some particular hogs (like, purely hypothetically speaking, GEOM threads)?

2. Why don't we change the default (for PREEMPTION and !FULL_PREEMPTION case) to
preempt_thresh=PRI_MIN_IDLE?  Plus sides of this have been reported via anecdotes.
 What down sides could there be?

Unfortunately somehow I just couldn't grasp ULE priorities and preemption, so I'd
like to ask for help of those who already have understood these things.

Thank you.
-- 
Andriy Gapon