Duration of Blocked Interrupts

Tue Apr 28 08:46:17 PDT 1998

(Skim the thread to the comment if you are on linux-smp or linux-tulip)

On Mon, 27 Apr 1998, Mail Receiver wrote:

> 
> We're doing some research into what parts of the Linux kernel block
> interrupts for the longest periods of time (as part of a real-time   
> project).  Surprise, surprise, we found the disk subsystem to be one of
> the biggest contributors.  In particular, we've found that the aic7xxx_isr
> routine can block interrupts for periods of as long as 400+ microseconds
> (of the other subsystems, they rarely kept interrupts disabled for more
> than 30-50 microseconds).  The average duration of interrupt blocking
> in the SCSI driver was on the order of 6 microseconds (whereas the other
> subsystems were on the order of 0.5 to 2 microseconds on average).
> 
> We're interested in trying to minimize this in one of two ways.  For
> certain processes, we'd like to either block their access to the driver
> (by queueing them up upon entry and leaving them blocked there until we
> decide to let them through) or limit the number of outstanding requests
> they can have (again, by blocking them temporarily).  Alternatively, if
> there are any ways to minimize the number of passes through the main loop
> in the ISR, we'd like to investigate them.  We're willing to sacrifice
> some of the performance for predictability (especially up front, we can
> work on optimizing it later). 
> 
> Any pointers or suggestions would be greatly appreciated.
> 
> Thanks for your time.
> 
> Robert
> 

On Mon, 27 Apr 1998, Matthew Jacob wrote:

> Can you clarify which release of the AIC code you were
> instrumenting? 
> 
> BTW- this is something I'm really interested in hearing as
> I've been running into some issues that this may have been
> caused by.

Actually, this sounds like it could be responsible for a lot of the
problems folks have had on systems with multiple, fast processors, fast
network adapters, and AIC SCSI adapters (such as mine:-).  400
microseconds is quite a long time on a heavily driven 100BT network, and
back when I was using the network drivers with "bad" SMP (non-atomic)
interrupt lock sets I noticed a very definite correlation between reads
from relatively slow devices (e.g. -- a SCSI CD-ROM) and a network
deadlockup.  A lot of folks have reported and are still reporting
bizarre problems on systems with AIC cards (although mine is now totally
stable thanks to the network driver fix).

Even though my system is "stable", it could be taking quite a
performance hit, and it would be very interesting to compare the
coincidence of this sort of interrupt blocking and the "Too much work
at..." messages in fast ethernet network drivers that occur when
interrupts are stacked too deep when the driver is finally entered.

In another message I didn't include (to keep the message finite in
length) it was suggested that the interrupt handler might should arrange
to do the blocking work in the bottom half handler with interrupts
enabled.  This sounds great, but has to work with SMP and with stacked
commands to the SCSI device that might come from different
processes/processors with some idea of sequence (so my one process
cannot somehow read or write twice in sequence to an open file and have
the second read/write BHH complete first because of scheduling
coincidences).  Just grepping on bottom, Bottom, BOTTOM, and half in
drivers/scsi/*.c suggests that not a lot of scsi drivers use bottom half
handlers.  Is this true?  Should this be true?  Seems like this could be
an immediate and significant performance boost and might resolve some
longstanding and annoying problems on SMP systems...

   rgb

Robert G. Brown	                       http://www.phy.duke.edu/~rgb/
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone: 1-919-660-2567  Fax: 919-660-2525     email:rgb at phy.duke.edu

To Unsubscribe: send mail to majordomo at FreeBSD.org
with "unsubscribe aic7xxx" in the body of the message