PLEASE REVIEW: Adding a pci_if method to facilitate specialized PCI bus drivers

[Thomas Moestl]

On Fri, 2003/06/27 at 14:50:08 -0400, John Baldwin wrote:
> On 27-Jun-2003 Thomas Moestl wrote:
> > On Fri, 2003/06/27 at 14:01:45 -0400, John Baldwin wrote:
> >> On 27-Jun-2003 Thomas Moestl wrote:
> >> > On Fri, 2003/06/27 at 13:37:00 -0400, John Baldwin wrote:
> >> >> On 13-Jun-2003 Thomas Moestl wrote:
> >> >> > This requires us to get this firmware property in the OFW PCI bus
> >> >> > driver before routing the interrupt; that can't be done in the pcib
> >> >> > route_interrupt method, since we don't know whether we are routing for
> >> >> > another bridge (where we use whichever index we get passed) or for a
> >> >> > child device (in which case we would need to look at the firmware
> >> >> > property).
> >> >> 
> >> >> Actually, can't you tell this by doing:
> >> >> 
> >> >>         if (device_get_parent(device_get_parent(dev)) == pcib)
> >> >>                 /* Routing direct child. */
> >> >>         else
> >> >>                 /* Routing descedent of a child bridge. */
> >> > 
> >> > No, pcib will always be a grandparent of dev. When routing a
> >                                                             ^ for 
> >> > descendant child bridge, dev will itself be the device_t of a bridge,
> >               ^ of a (oops)
> >> > otherwise it is that of the device we are routing to.
> >> Doh, yes. :(  Hmm, can you try something like this maybe:
> >> 
> >>         if (pci_get_class(dev) == PCIC_BRIDGE &&
> >>             pci_get_subclass(dev) == PCIS_BRIDGE_PCI)
> >>                 /* Routing across a child bridge. */
> >>         else
> >>                 /* Routing a direct child that is not a bridge. */
> > 
> > This leaves two possible problems: first, there are other types of
> > bridges (we currently support PCI-ISA and PCI-EBus ones, cardbus might
> > also work) for which we need to use PCIB_ROUTE_INTERRUPT(); that could
> > likely be dealt with by not testing the subclass at all.
> > More importantly, however, a bridge might want to allocate an
> > interrupt for itself; for example, cardbus bridge drivers do this to
> > handle insertion/ejection/etc events.
> > 
> > I think that handling this at the bus driver level is a much cleaner
> > solution.
> 
> Then should we eliminate pcib_route_interrupt() and make it a PCI bus
> method?  I just think it is rather odd to require this interface to be
> duplicated in two different places.

This is not necessarily duplication; in the OFW_PCI code, the work to
be done is divided up between the two.
Where the routing has to happen depends a lot on the information
available. If a mapping from device to interrupt is available, it
can make sense to do the routing completely in the bus driver. If
there are any bridges for which we lack such information, however, and
we need to fall back to other routing methods (like swizzling), there
might be a need to support the bridge routing methods. In such a
scenario, the bus and bridge methods usually would be quite different,
so there would be not much duplication.

To give an example, OpenFirmware contains an 'interrupt' property for
each PCI device which uses interrupts. This property can contain a
full interrupt number, or an index which has to be mapped using the
'interrupt-map' properties of higher bridges. In some cases (but not
always), these indices are actually intpins, and on some bridges we
need to do a swizzle instead of a map lookup.
We look up the 'interrupt' property of the device in the bus driver,
and decide whether it is full interrupt number already, in which case
we just use it. Otherwise, we let the parent bridge do the mapping as
required (which in turn might need to resort to it's parent bridge,
and so on).
This scheme also allows us to interface with PCI bridge drivers which
use their intpins, and for which we do not have (or need)
OpenFirmware-specific drivers.
Doing all of that in the bus method would be a layering violation, and
would have ugly consequences (for example, different bridges need to
handle different quirks).

> Doing it in the bus might actually make life simpler.  Right now
> when using different tables for routing PCI interrupts on x86, I
> have to write two different bridge drivers all 
> the time, one for host bridges and one for PCI-PCI bridges, thus we have
> ACPI host-PCI and PCI-PCI bridge drivers, PCIBIOS $PIR host-PCI and PCI-PCI
> bridge drivers and MPTable host-PCI and PCI-PCI bridge drivers.  Being
> able to just have an ACPI PCI bus driver, a PCIBIOS PCI bus driver, and
> an MPTable PCI bus driver would be nice.

I don't know much about these tables, so I don't know how well this
would work. Is it guaranteed that all bridges will have
ACP/PCIBIOS/MPTable tables, and generic drivers will never be used?

At least the PCIBIOS method seems to just offer routing as far as I
can see (no special enumeration, and no extra per-device information
to be considered), so having a bus driver for it would probably be
overkill, since it can be handled well in a bridge driver. Needing an
extra one for host bridges is a bit annoying, but most of the code
could be shared, and it is probably much worse to not be able to
interact with the generic drivers any more.

> However, I am curious what pushing this down into the bus layer buys
> you.  Do you have the sparc64 PCI bus and PCI bridge drivers up
> somewhere?

The patch to implement OFW_NEWPCI is at
  http://people.freebsd.org/~tmm/ofw-newpci2.diff
(it has two conflicts right now, I'm just merging it up). It contains
both drivers, among other changes.

	- Thomas

-- 
Thomas Moestl <t.moestl at tu-bs.de>	http://www.tu-bs.de/~y0015675/
              <tmm at FreeBSD.org>		http://people.FreeBSD.org/~tmm/
PGP fingerprint: 1C97 A604 2BD0 E492 51D0  9C0F 1FE6 4F1D 419C 776C