can someone explain...[ PCI interrupts]

[John Baldwin]

On Wednesday 07 December 2005 02:38 am, Julian Elischer wrote:
> > have been used already, then  the kernel starts assigning multiple
> > (apic, pin) tuples to the same  IRQ resulting in interrupts being shared
> > in software because of the  cpl limitation even though they aren't
> > shared in hardware.  This is  why your IRQ values are different on 4.x
> > than on FreeBSD 5.2+ and  Linux which use the ACPI global interrupt
> > number model.
>
> but if I change the code that does this, I may be able to get my devices
> that collide with the 'boot interrupt' to go elsewhere? That would be
> good..

No, probably not.  The "boot interrupt" collisions happen on all versions of 
FreeBSD currently.  I do have a workaround in my head, and if it works, it 
might even be backportable to 4.x.  You can't change how the interrupts are 
physically wired though, and the boot interrupt collisions happens because of 
issues in hardware.  You might be able to pull a trick where you map the two 
colliding interrupts to the same IRQ cookie on 4.x, but that'd be ugly, and 
the fix I'm considering would be a lot simpler and do the same thing (I need 
to check, but I think that the INTx swizzle the PXH's do might match the 
standard PCI-PCI bridge swizzle, and if so, we can just depend on the boot 
interrupt and route the interrupts via the boot destination by ignoring the 
_PRT (for ACPI) on such bridges, and ignoring any MP Table entries (on 
non-ACPI) so that it falls back to using the PCI-PCI swizzle.

> > for the (apic, pin) tuple being used.  (Thus, IRQs are  just a cookie
> > that is the index into the global array of interrupt  sources on x86.)
> > Note that interrupts routed this way are hardwired  into the motherboard
> > design.  There's no chance for the OS to change  which (pic, pin) a PCI
> > device interrupt is hooked up to.
>
> but from my memory, many PCI devices can select between A,B,C and D
> so maybe by going to the device and selecting a different one of those
> you can force it to go elsewhere...

They devices don't really get to choose, it's a read-only config register that 
is set in silicon.  Even then, IIRC, PCI mandates that single-function 
devices use INTA, and that multi function devices use INTA if they have one 
interrupt, INT[AB] if they have two, etc.  (I'm less certain about the 
multifunction part, but single-function devices must use INTA.)

> > already.  If so, that's the IRQ that that PCI device interrupt is
> > assigned to.  If an IRQ isn't routed already, then it has to use an
> > algorithm to pick one, make a BIOS call to route the link to the  chosen
> > IRQ, and then assign the PCI device interrupt to that IRQ.
>
> so, is a "link device" a physical piece of hardware or a software
> abstraction?

It's a physical piece of hardware in that it represents a pin on a 
programmable interrupt router.  You basically have a chip that has several 
input pins (each of which is a link device) and the chip can programmably 
route each intput pin to one of several output pins.  Thus, you might have a 
single chip but with multiple pins (like an APIC with 24 different pins) and 
each input pin is considered a link device.

> > Hopefully this at least answers some questions and gives a good
> > overview of what PCI interrupt routing is and how it works, etc.
>
> My head hurts, but a lot makes more sense now.
> I'll need to read this a few more times however.
> if you made this into a web page, and added a few diagrams that would be
> amazing.. also you use a few Acronyms without saying what they are..

Yeah, I should probably put this in the arch-handbook, but I'd need to learn 
pic to draw the diagrams (or perhaps I could draw them in something else and 
export it as .eps?)

-- 
John Baldwin <jhb at FreeBSD.org>  <><  http://www.FreeBSD.org/~jhb/
"Power Users Use the Power to Serve"  =  http://www.FreeBSD.org