couple of sched_ule issues

[Andriy Gapon]

This is more of a "just for the record" email.
I think I've already stated the following observations, but I suspect that they
drowned in the noise of a thread in which I mentioned them.

1. Incorrect topology is built for single-package SMP systems.
That topology has two levels ("shared nothing" and "shared package") with exactly
the same CPU sets.  That doesn't work well with the rebalancing algorithm which
assumes that each level is a proper/strict subset of its parent.

2. CPU load comparison algorithms are biased towards lower logical CPU IDs.
With all other things being equal the algorithms will always pick a CPU with a
lower ID.  This creates certain load asymmetry and predictable patterns in load
distribution.

Another observation.
It seems that ULE makes a decision about thread-to-CPU affinity at the time when a
thread gets switched out.  This looks logical from the implementation point of
view.  But it doesn't seem logical from a general point of view - when the thread
will be becoming running again its affinity profile may become completely
different.  I think that it would depend on how much a thread actually spends not
running.

-- 
Andriy Gapon