cvs commit: src/sys/dev/aac aac_pci.c

[Scott Long]

[bouncing to arch@ to get more comments]

On Tue, 10 Feb 2004, Bruce Evans wrote:
> > scottl      2004/02/08 21:29:05 PST
> >
> >   FreeBSD src repository
> >
> >   Modified files:
> >     sys/dev/aac          aac_pci.c
> >   Log:
> >   Fall back to INTR_MPSAFE if INTR_FAST registration fails.
> >
> >   PR:     kern/62276
>
> This depends on losing the race to register the interrupt as INTR_FAST.
> sio has the same bug, but it is less serious there because most sio
> devices use unshared isa interrupts.  The cy driver avoids this bug
> by not trying INTR_FAST unless CY_PCI_FASTINTR is configured.  (The
> bug is mostly in bus_setup_intr().  Drivers cannot know at their attach
> time (or even later) whether the interrupt will be shared later.  The
> sharedness may change back and forth as devices and drivers are plugged
> in and out.)

And thus we come back to the argument for having two-tiered interrupt
handling, with a driver having the freedom to choose which of either, or
both, to register against.  Interrupt sharing is an unfortunate reality,
but should we discourage drivers from being fast merely because they
can't intelligently choose which way to register?  INTR_FAST handling
shows a significant win in the case of aac, and makes a compelling
argument for other drivers to be made INTR_FAST.  It's also been shown
time and again that 5-current suffers in all respects from high
interrupt latency.  So arbitrarily saying that driver X, which generates
maybe 100 int/sec at it's peak, is of higher priority in the
registration scheme than driver Y which generates 10,000 int/sec, seems
rather odd.

What I'd really like to see is the following:

- non-exclusive fast handlers that allow a driver to quickly ACK the
   hardware and schedule the appropriate completion mechanism.
- taskqueues be deprecated from being a general completion mechanism
   to being an exception-only or low-priority mechanism.
- ithreads be allowed to be scheduled from a fast handler to handle
   completion processing.  Ideally ithreads should be dedicated, but I'm
   not sure if this is feasable.
- if no fast handler is registered, then the registered ithread handler
   is scheduled to service the interrupt.  This allows drivers to
   transition at their leasure, and keeps support for things like cardbus
   that likely can never be made INTR_FAST.
- callbacks into the driver when a fast handler cannot be granted or
   needs to be demoted for whatever reason.

While the argument can be made to concentrate solely on making ithread
scheduling really fast, that strategy means that holding a lock in an
ithread will prevent other hardware on the shared interrupt from
getting serviced until the lock is released.  I'm all for making
ithreads be really fast, though, and I think that it would be a good
compliment to the two-tier strategy.


Scott