FreeBSD memguard + spinlocks
John Baldwin
jhb at freebsd.org
Mon Apr 13 17:55:26 UTC 2009
On Saturday 11 April 2009 5:03:58 pm Andrew Brampton wrote:
> 2009/4/11 Robert Watson <rwatson at freebsd.org>:
> > On Sat, 11 Apr 2009, Andrew Brampton wrote:
> >
> > Your understanding is mostly right. The missing bit is this: there are
two
> > kinds of interrupt contexts -- fast/filter interrupt handlers, which
borrow
> > the stack and execution context of the kernel thread they preempt, and
> > interrupt threads, which get their own complete thread context.
> >
> > Fast interrupt handlers are allowed unlock to acquire spinlocks so as to
> > avoid deadlock because of the borrowed context. This means they can't
> > perform any sort of sleep, or acquire any locks that might sleep, since
the
> > thread they've preempted may hold conflicting locks, or be the one that
> > would have woken up the sleep that the handler performed. Almost no code
> > will run in fast handlers -- perhaps checking some device registers, doing
> > work on a lockless or spinlock-protected queue, and waking up a worker
> > thread.
> >
> > This is why, BTW, spin locks disable interrupt: they need to control
> > preemption by other interrupt handlers to avoid deadlock, but they are not
> > intended for use except when either in the scheduler, in a few related IPI
> > contexts, or when synchronizing between normal kernel code and a fast
> > handler.
> >
> > Full interrupt thread contexts are permitted to perform short lock sleeps,
> > such as those performed when contending default mutexes, rwlocks, and
> > rmlocks. They are permitted to invoke kernel services such as malloc(9),
> > UMA(9), the network stack, etc, as long as they use M_NOWAIT and don't
> > invoke msleep(9) or similar unbounded sleeps -- again to avoid the
> > possibility of deadlocks, since you don't want an interrupt thread
sleeping
> > waiting for an event that only it can satisfy.
> >
> > So the first question, really, is whether you are or mean to be using
> > fast/filter interrupt handler. Device drivers will never call memory
> > allocation, free, etc, from there, but will defer it to an ithread using
the
> > filter mechanism in 8.x, or to a task queue or other worker in 7.x and
> > earlier. If you're using a regular INTR_MPSAFE ithread, you should be
able
> > to use only default mutexes (a single atomic operation if uncontended)
> > without disabling interrupts, etc.
> >
> > Robert N M Watson
> > Computer Laboratory
> > University of Cambridge
> >
>
> Anyway, that is why I also asked about a lighter weight spin lock
> (perhaps similar to this one). I tempted to replace this custom
> spinlock with the standard MTX_DEF, however I'm unsure of its impact.
> The custom spin lock seems quick and light to acquire, and it does not
> concern me that a interrupt can potentially interrupt the code.
You should just use a MTX_DEF mutex. Also, if you use M_NOWAIT, you will need
to handle malloc() returning NULL. In general I try to allocate things while
not holding any locks when possible and only acquire the lock to initialize
the memory returned from malloc().
--
John Baldwin
More information about the freebsd-hackers
mailing list