FreeBSD spinlock - compatibility layer

[Alfred Perlstein]

On 5/22/13 9:05 AM, John Baldwin wrote:
> On Tuesday, May 21, 2013 4:40:16 pm Alfred Perlstein wrote:
>> On 5/20/13 9:50 AM, John Baldwin wrote:
>>> On Tuesday, May 14, 2013 6:04:21 am Orit Moskovich wrote:
>>>> Hi,
>>>>
>>>> I read about the FreeBSD mutex implementation for spinlock in the
>>> compatibility layer.
>>>> I might be wrong, but I noticed a code section that might be problematic:
>>>>
>>>> Taken from
> http://svn.freebsd.org/base/release/9.1.0/sys/ofed/include/linux/spinlock.h:
>>>> static inline void
>>>> spin_lock_init(spinlock_t *lock)
>>>> {
>>>>
>>>>           memset(&lock->m, 0, sizeof(lock->m));
>>>>           mtx_init(&lock->m, "lnxspin", NULL, MTX_DEF | MTX_NOWITNESS);
>>>> }
>>>>
>>>> But MTX_DEF initializes mutex as a sleep mutex:
>>>>
>>>> By default, MTX_DEF mutexes will context switch when they are already
>>>>
>>>>        held.
>>>>
>>>>
>>>> There is a flag MTX_SPIN Which I think is the right one in this case .
>>>>
>>>>
>>>>
>>>> I'd appreciate your take on this issue.
>>> Since FreeBSD uses a different approach to interrupt handlers (they run in
>>> threads, not in the bottom half), a regular mutex may in fact give the
> closest
>>> match to the same semantics.  Regular mutexes are also cheaper and in
> general
>>> preferable to spin mutexes whenever possible.
>>>
>> Sure, but is it possible that someone might want some of the other
>> guarantees of MTX_SPIN spinlocks such as:
>>
>> critical section/non-pre-emptable/non-migrating on cpu/latency versus
>> throughput ?
> Probably not.  For example, on FreeBSD you want your driver lock to be
> preempted by an interrupt to avoid higher interrupt latency for filter
> handlers.  Most drivers should not need temporary pinning.  If they want to
> pin work to threads they should bind threads or IRQs to specific CPUs, not
> rely on temporary pinning.
>
I know how it works in FreeBSD.

I think that a compatibility layer should first and foremost aim for 
compatibility, not speed at expense of expected semantics.

That said, I'm hopeful that the ofed stack doesn't use any of the other 
guarantees you'd expect from real spinlocks other than mutual exclusion 
so it's not that big of a deal.  Unless it does... which will be 
interesting track down.

  -Alfred