FreeBSD on Layerscape/QorIQ LX2160X

[Dan Kotowski]

> > > > In the old firmware, under the Root Complex Nodes, `Output reference: 0x30` points to the ITS Node
> > > > with node offset 0x30.
> > > > Then in the new firmware, under the Root Copmlex Nodes, `Output reference: 0x48` points to the SMMU
> > > > with node offset 0x48.
> > > > Is this what you meant when you said everything is "behind the SMMU"?
> > > > Yes.
> > > > What patches are we applying right now? I think some new builds would be good, including one with
> > > > all the stuff we've fixed - like AHCI - but with the NXP PCIe code so I can test on the old
> > > > firmware. If I'm keeping track correctly, this includes:
> > > >
> > > > -   D20835: enable tagged pointers
> > > > -   D20974: Port sbsawdt drive from NetBSD
> > > > -   D21017: armv8crypto: add AES-XTS support
> > > > -   D24423: arm/pmu: add ACPI attachment, more FDT names
> > > >
> > > > These are not directly related to running on NXP, just random improvements :)
> > > >
> > > > -   D25145: acpi_resource: support multiple IRQs
> > > > -   D25157: ahci_generic: add quirk for NXP0004
> > > > -   D25179: acpi_iort: fix mapping end calculation
> > > >
> > > > Yes, these three.
> > > > Here's the pci_layerscape patch:
> > > > https://github.com/DankBSD/base/commit/c1ea44aa33b29f74daed89eee82b3dfeb105d376.patch
> > > > If we haven't tried it + acpi_iort fix before, which is quite likely, maybe that's a combination
> > > > that would work.
> > > > (I remember when we tried it only, there was only the same interrupt problem as usual..)
> > > > It's honestly kinda weird that "old" FW requires the custom controller access while "new" FW
> > > > requires not doing it >_<
> > >
> > > Any tips on where to add verbosity to dmesg during boot? You had a handful of extras in there to
> > > print things things from the IORT.
> >
> > That thing is here:
> > https://github.com/freebsd/freebsd/blob/6cee1596c05e8a9ab64812444627b61c584ca6bc/sys/arm64/acpica/acpi_iort.c#L169
> >
> > > Also seeing that NVMe hangs in nvme_ctrlr_start_config_hook,
> > > maybe we want to add some verbosity to sys/dev/nvme/nvme_ctrlr.c to see what it's waiting for?
> >
> > We know it's an interrupt.. everything is waiting for an interrupt.
>
> Well, at least we're stable enough to buildworld and buildkernel on-system! :partyparrot:
>
> Waiting on a new SSD, but building on a USB uSD:
>
> ----------------------------------------------------------------------------------------------------------------------------------------------
>
> > > > World build completed on Wed Jul 8 02:47:59 UTC 2020
> > > > World built in 6897 seconds, ncpu: 16, make -j16
>
> Forgot to copy off the buildkernel time, but it was just under 900 seconds.
>
> Interestingly, we have real temp readouts from thermal zones, which is new. Did I add something back in that we had previously removed?

Here's something: https://gist.github.com/agrajag9/11efe00514513100232999e0a9ec612c

NVMe is working. It mapped an interrupt. I have no idea what changed to make this work other than compiling my own kernel. I applied the 7 patches above and added a bunch of my own debug printfs (dmesg.boot lines beginning with A9DEBUG), but it's the same POC ECAM firmware we've been using for a while. I'll test with the HBA later this week - it's currently in use elsehwere in my homelab doing zfs send|receive things.