Question about interrupt threads

[Bruce Evans]

On Tue, 22 Apr 2003, Aniruddha Bohra wrote:

> Reading the implementation of interrupt handling on i386,
> and reading Greg Lehey's Usenix paper on FreeBSD 5.0 SMP
> implementation, I understand that interrupt handling is done in
> process context. This is needed partly to support sleeping while
> handling the interrupt because GIANT needs to be locked and
> a sleep is possible there.
>
>    My question is after all subsystems that lock giant are made
> INTR_MPSAFE will the implementation go back to non-process
> context interrupt handling (as in older versions) ?

I would like that, but don't see how it could work right.

>    Moreover, will the locking be per-IRQ or there will be a
> global IRQ lock for ensuring atomic access to the ISRs ?

Neither.  Each interrupt handler would lock whatever it accesses, and
hopefully not much more, and not too often (zillions of locks and
unlocks per interrupt), and without much lock contention and without
any deadlocks.  Locking using Giant basically locks everything, which
is far too much.  One advantage of Giant locking is that it tends to
require only 1 lock/unlock per interrupt, but I think that advantage is
small compared with the disadvantages of extra lock contention (almost
everything competes for Giant, and switches to an interrupt handler that
does nothing at first except block on Giant and switch back are common).
Otherwise, Giant locking of interrupt handlers is little different from
correct locking (if any).  Both require a context to hold the state while
blocked on a (non-spin) lock.  In -current, that context is an ordinary
thread context which is too heavy weight, but this is simplest and good
for debugging.

>    Finally why do we need to create a kernel thread for
> each IRQ - why can't one kernel thread handle all the
> IRQs ?

First, multiple threads are useful because they can be prioritized and
scheduled independently.  This is probably not very important for
interrupts provided there is no slow PIO hardware for critical devices
so that no interrupt handler wants more than a few usec at a time (just
run the critical devices round-robin and defer processing for the
non-critical ones to timeout routines).

Second, multiple threads are needed so that they can block on locks
independently.  I think we actually use 1 thread per device driver to
irq attachment, so we have multiple threads per irq in the shared irqs
case.  This is probably needed to let independent device drivers of
device that are attached to the same irq block independently.  We don't
seem to have any examples of 1 thread per device giving multiple threads
per irq withing 1 driver, perhaps because few drivers are fully SMP-aware
internally.

I would like at most 1 kernel interrupt thread per CPU, but don't see
how it could work right.

Bruce