Minor ULE changes and optimizations

[Harrison Grundy]

On 02/28/15 04:24, John Baldwin wrote:
> On Friday, February 27, 2015 07:50:55 AM Harrison Grundy wrote:
>> On 02/27/15 06:14, John Baldwin wrote:
>>> On Thursday, February 26, 2015 06:23:16 AM Harrison Grundy
>>> wrote:
>>>> https://reviews.freebsd.org/D1969 This allows a
>>>> non-migratable thread to pin itself to a CPU if it is already
>>>> running on that CPU.
>>>> 
>>>> I've been running these patches for the past week or so
>>>> without issue. Any additional testing or comments would be
>>>> greatly appreciated.
>>> 
>>> Can you explain the reason / use case for this?  This seems to
>>> be allowing an API violation.  sched_pin() was designed to be
>>> a lower-level API than sched_bind(), so you wouldn't call 
>>> sched_bind() if you were already pinned. In addition,
>>> sched_pin() is sometimes used by code that assumes it won't
>>> migrate until sched_unpin() (e.g. temporary mappings inside an
>>> sfbuf).  If you allow sched_bind() to move a thread that is
>>> pinned you will allow someone to unintentionally break those
>>> sort of things instead of getting an assertion failure panic.
>> 
>> For a pinned thread, the underlying idea is that if you're
>> already on the CPU you pinned to, calling sched_bind with that
>> CPU specified allows you to set TSF_BOUND without calling
>> sched_unpin first.
>> 
>> If a pinned thread were to call sched_bind for a CPU it isn't
>> pinned to, it would still hit the assert and fail.
>> 
>> For any unpinned thread, if you're already running on the correct
>> CPU, you can skip the THREAD_CAN_MIGRATE check and the call to
>> mi_switch.
> 
> Ah, ok, so you aren't allowing migration in theory.  However, I'm
> still curious as to why you want/need this.  This makes the API
> usage a bit more complex to reason about (sched_bind() can
> sometimes be called while pinned but not always after this change),
> so I think that extra complexity needs a reason to exist.

Primarily, it allows those threads already on a CPU to skip the call
to mi_switch and get out of sched_bind a bit faster.

Additionally, it allows a driver to call sched_pin, then bind to that
same cpu later without having to write something like
"critical_enter(); sched_unpin(); sched_bind(foo, bar);
critical_exit();", since otherwise it could be migrated/preempted
between unpin and bind.