Potential source of interrupt aliasing

[John Baldwin]

On Sunday 10 April 2005 02:22 am, Doug White wrote:
> On Fri, 8 Apr 2005, John Baldwin wrote:
> > > A quick hack would be to blacklist intpin 16 on the first IOAPIC and
> > > bump any PCI devices that ACPI says are claiming that interrupt. I
> > > don't know how difficult this is to do with ACPI.  How to handle this
> > > for ATPIC mode is a little difficult since the boot interrupt either
> > > gets routed to IRQ9 or ends up as a stray on IRQ7 (on my SCB2 at least
> > > -- other boards may vary).
> >
> > You can't just move APIC interrupts around.  They are physically wired
> > that way.  ATPIC mode won't matter.  If there is a way to disable this in
> > hardware, then it needs to be done in the _PIC method of the BIOS for
> > ACPI. I don't think there's any way to allow for this for non-ACPI though
> > as the MPTable spec only mentions the IMCR which these boards don't
> > implement.
>
> I didn't think so, at least in the general case. It may be possible to
> reprogram ICH registers to have usb use other INT lines if they're coming
> up through SERIRQ and aren't physically wired to the pin, but I haven't
> dug through the docs for that.
>
> For fun, I put together this one-liner to at least test the theory:
>
> http://people.freebsd.org/~dwhite/intr_machdep.c.20050409.patch

Doing this should result in an instant interrupt storm since your PCI 
interrupts (which are level triggered) should now keep firing over and over 
and the machine will spend all it's time rescheduling the ithread.  If it's 
not doing that then this is very curious.

> This makes the aliased interrupts go away at the cost of hitting
> sched_lock for each ithread interrupt.  I'm trying to dream up a way of
> telling if the ithread is active or scheduled so we don't need to bump it
> again with setrunqueue().  I think I can use the it_need sparkplug to at
> least save mutex grabs on interrupts that fire much faster than their
> ithread handlers cycle.
>
> I'd like better granularity, though, but I'm not sure that setting a flag
> that is cleared just before mi_switch() in ithread_loop() and set just
> after won't race against an interrupt checking it to see if it needs to
> setrunqueue() the ithread. Maybe clear the flag before trying to grab
> sched_lock?

it_need already does this.  You haven't added any extra sched_locks.  What've 
you done is change the interrupt code to never mask interrupt sources.  If 
you aren't getting an interrupt storm on your first PCI interrupt then that 
is very curious indeed.  Have you tried this with SMP disabled so that you 
only have one CPU running?  It may be that due to SMP your ithread still gets 
a chance to run on one CPU while the other CPU is stormed with interrupt 
requests.  Note that since storm detection is done in the ithread, this type 
of storm won't be detected and throttled.  It also might not show up in the 
vmstats since we don't use atomic ops there anymore depending on whether or 
not the interrupt bounces between CPUs that keep overwriting the count on top 
of each other.

> I'm also not sure how much this change affects system performance. I guess
> I should hit up rwatson for some info on MUTEX_PROFILING. :)

First see how the system acts with only one CPU.

-- 
John Baldwin <jhb at FreeBSD.org>  <><  http://www.FreeBSD.org/~jhb/
"Power Users Use the Power to Serve"  =  http://www.FreeBSD.org