Should the MPC8544 root complex really show up as a device?

[Justin Hibbits]

On Mon, Apr 13, 2015 at 3:39 PM, Marcel Moolenaar <marcel at xcllnt.net> wrote:
>
>> On Apr 13, 2015, at 3:28 PM, Justin Hibbits <jrh29 at alumni.cwru.edu> wrote:
>>
>> I'm trying to get the mini-PCIe slot on my RouterBoard RB800 working,
>> and running into a snag.  It works just fine in RouterOS (Linux), but
>> on FreeBSD, the card in the slot doesn't show up.  From all my
>> Googling, it appears the slot should show up on PCIe controller 1, but
>> the only device that shows up is the root complex as 1/0:0:0, as type
>> Processor.  Should this show up anyway?  Shouldn't the device behind
>> it show up instead?
>
> The problem is that the device is flagged as a processor, not
> as a PCI bridge. That’s why it shows up and also why you don’t
> find anything underneath. One way to fix it is by spoofing the
> device type in sys/powerpc/mpc85xx/pci_mpc85xx.c or otherwise
> by changing common PCI code to accept this device as a (host)
> bridge.
>
> HTH,
>
> --
> Marcel Moolenaar
> marcel at xcllnt.net

Thanks Marcel,

I did try spoofing it in pci_mpc85xx.c, with the attached patch (patch
includes quite a bit of debug as well).  The reference manual doesn't
indicate if the root complex appears on every controller, or just the
first (there are 3 PCIe controllers on the MPC8544, and only the first
has this device showing up).  Maybe I'm missing something.  This is
new to me, and PCI configuration has always baffled me.  Trying to
read bus 1 on the controller (which is configured in the Second Bus
register), I only saw 0xff returned during probing in fsl_pcib_init().
Been looking at the Linux driver for hints, but not a whole lot has
come from that.

Unfortunately, the last couple days my RouterBoard has decided it
doesn't want to boot FreeBSD, so I have to debug that before I can
continue working on this, but PCI register 0x4B0 (CFG_READY) seems a
contender for needing initialized, although the documentation doesn't
indicate if it's only relevant for EP mode or also relevant for RC
mode.

- Justin
-------------- next part --------------
Index: sys/powerpc/mpc85xx/pci_mpc85xx.c
===================================================================
--- sys/powerpc/mpc85xx/pci_mpc85xx.c	(revision 281541)
+++ sys/powerpc/mpc85xx/pci_mpc85xx.c	(working copy)
@@ -269,6 +269,7 @@
 
 	if (sc->sc_pcie) {
 		ltssm = fsl_pcib_cfgread(sc, 0, 0, 0, PCIR_LTSSM, 1);
+		printf("PCI %d: LTSSM = %x\n", device_get_unit(dev), ltssm);
 		if (ltssm < LTSSM_STAT_L0) {
 			if (bootverbose)
 				printf("PCI %d: no PCIE link, skipping\n",
@@ -445,6 +446,19 @@
 	}
 }
 
+static void
+fsl_pcib_init_fsl(struct fsl_pcib_softc *sc, uint16_t device, int bus,
+    int slot, int fn)
+{
+	uint16_t val;
+
+	val = fsl_pcib_read_config(sc->sc_dev, bus, slot, fn, 0x44, 2);
+
+	fsl_pcib_write_config(sc->sc_dev, bus, slot, fn, 0x44, val | (1 << 10), 2);
+	fsl_pcib_write_config(sc->sc_dev, bus, slot, fn, PCIR_LATTIMER, 0, 1);
+	fsl_pcib_write_config(sc->sc_dev, bus, slot, fn, 0x4b0, 1, 4);
+}
+
 static int
 fsl_pcib_init_bar(struct fsl_pcib_softc *sc, int bus, int slot, int func,
     int barno)
@@ -516,12 +530,15 @@
 	uint8_t command, hdrtype, class, subclass;
 
 	secbus = bus;
+	printf("Checking bus %d maxslot: %d\n", secbus, maxslot);
 	for (slot = 0; slot <= maxslot; slot++) {
 		maxfunc = 0;
 		for (func = 0; func <= maxfunc; func++) {
+			printf("Reading %d/%d/%d\n", bus, slot, func);
 			hdrtype = fsl_pcib_read_config(sc->sc_dev, bus, slot,
 			    func, PCIR_HDRTYPE, 1);
 
+			printf("Read hdrtype: %x\n", hdrtype);
 			if ((hdrtype & PCIM_HDRTYPE) > PCI_MAXHDRTYPE)
 				continue;
 
@@ -537,6 +554,9 @@
 				sc->sc_devfn_tundra = DEVFN(bus, slot, func);
 				continue;
 			}
+			if (vendor == 0x1957 && device == 0x33) {
+				fsl_pcib_init_fsl(sc, device, bus, slot, func);
+			}
 
 			command = fsl_pcib_read_config(sc->sc_dev, bus, slot,
 			    func, PCIR_COMMAND, 1);
@@ -570,6 +590,16 @@
 			subclass = fsl_pcib_read_config(sc->sc_dev, bus, slot,
 			    func, PCIR_SUBCLASS, 1);
 
+			/* Quirk: Root complex is a bridge. */
+			if (hdrtype == PCIM_HDRTYPE_BRIDGE) {
+				printf("Treating BDF %d/%d/%d class %x:%x as a bridge\n",
+					bus, slot, func, class, subclass);
+				class = PCIC_BRIDGE;
+				subclass = PCIS_BRIDGE_PCI;
+				fsl_pcib_write_config(sc->sc_dev, bus, slot, func, PCIR_CLASS, class, 1);
+				fsl_pcib_write_config(sc->sc_dev, bus, slot, func, PCIR_SUBCLASS, subclass, 1);
+			}
+
 			/* Allow only proper PCI-PCI briges */
 			if (class != PCIC_BRIDGE)
 				continue;