Weird PCI interrupt delivery problem

[Craig Boston]

Hi,

I'm working on getting this laptop up and running and need some advice
from PCI gurus.  I am running into a really odd problem with PCI
interrupts.  After a while they simply stop being delivered.  ACPI makes
the problem much worse, but it happens eventually without ACPI too.

The system looks like this:
pcib0
  pci0
    ohci0
  pcib2
    pci9
      cbb0
      rl0
      ath0

However, the problem affects ohci0 as well so I don't think the PCI
bridge is the culprit.  Actually, the only PCI device in the system that
doesn't seem to be affected is the ATA controller, and I think that's
because it uses ISA interrupts 14-15.

With both ACPI & APIC enabled, it only lasts a few seconds.  Each pin on
the I/O APIC manages about 10-50 interrupts before they simply stop
coming.  The number of interrupts seems to be the deciding factor rather
than time -- I can wait a minute and ohci0 will work until I move a USB
mouse around for a while.

With ACPI disabled, the system panics because the mptable is broken.
However, I was able to hack the kernel to override the mptable and route
the interrupts to the correct pins (actually it rewrites parts of the
mptable as it's being parsed).  In this configuration, everything works
fine for a while, but it eventually dies.  ath0 is usually the first to
go since it generates a steady stream of interrupts, but given enough
time they eventually all stop.  Sometimes it happens after 50,000
sometimes 500,000.

I also tried ACPI enabled but APIC disabled.  The FreeBSD ACPI code
seems to assume APIC interrupt model for i386, so it took some
modifications to get this working.  Everything ends up on IRQ 11, though
I'm not sure if it's getting reprogrammed to be level triggered or not.
Symptoms are the same as with APIC on -- after 10-50 interrupts it just
stops.

The final thing I tried is both APIC & ACPI disabled -- route everything
through the 8259.  In this mode, cbb0 fails to attach (Unable to map
IRQ).  Everything else ends up on IRQ 11, however it does seem to work
indefinitely.

Oh, when APIC is being used, vmstat -i reports the lapic timer interrupt
happily churning away without problem.

I've checked everything I can think of -- no reports of interrupt
storms, everything looks normal in verbose boot.  I was just going to
run in PIC mode until I discovered that cardbus didn't work.

Any ideas on things to try to debug this?  First thing that comes to
mind is to see if the IRQ is being intentionally masked for some reason,
but I can't think of an easy way to check that.

Thanks,
Craig

P.S. Yes, it does sound like a wonky PCI bus, but stock XP runs with
ACPI & APIC with no problems.