FreeBSD on Layerscape/QorIQ LX2160X

[Dan Kotowski]

> > > > > By the way, did you get any different firmware builds in the meantime? That don't have everything suspiciously routed to the SMMU in IORT..
> > > >
> > > > s/suspiciously/by design/
> > > > BEGIN QUOTE
> > > > the PCIe root nodes are hidden from the rich OS with this configuration. To access the root nodes you need a quirk implemented.
> > > > eventually it will be an option for those that want to run custom kernels, but for now this is the solution for the most SBSA like configuration
> > > > END QUOTE
> > >
> > > And how are we supposed to get any MSI-X interrupts in this configuration?
> > > Our fallback code for when nothing was matched in IORT (i.e. directly using PCIe RID with no offset) did not result in working interrupts.
> > > IIRC legacy interrupts didn't work either, but maybe you should retest them (disabling MSI, MSI-X).
> >
> > NSTR, but here you go anyways: https://gist.github.com/agrajag9/d4d75d7dca41b3cb64c0a4243eed4eb7
> > Forgive my n00bishness and feel free to correct me below, I'm still learning my way around down here... What are we doing with all the GSIVs in the IORT SMMU node?
> > Based on reading the table and dmesg.boot, I'm seeing the following:
> >
> > -   ITS node for gics
> > -   2 root complex nodes for pci0 and 1,
> > -   Named component nodes for MCE, ugens, mmcs (if we still had that in), and ahcis
> >
> > But then the SMMU node has:
> >
> > -   64 context interrupts,
> > -   10 PMU interrupts, and
> > -   5 ID mappings
> >
> > Where do these go? Are we perhaps not walking this section properly and that's the quirk to which Jon is referring?
>
> We do not support the SMMU at all. (There is a patch for SMMUv3 support but this chip has a v1/v2.)
>
> SMMU support should not be mandatory, it's an IOMMU used for virtualization or additional DMA security protection (like dmar on Intel).
>
> NetBSD does not support it either, and they don't seem to have any interrupt problems..
>
> In public code (lx2160a master), PCIe interrupts are routed to the ITS after all.

But NetBSD did not have those interrupt problems on the older firmware, where the ECAM workaround was done via PCI quirks. I fetched a generic evbarm img to test with the current firmware and it looks like NetBSD may now be in the same boat as us.

https://gist.github.com/agrajag9/daa072b23dbb4cdb9dae899a5b2d01f5

Jon says there's a new build with published sources coming soon, supposedly with better ECAM support (current is "proof of concept"). Hopefully that helps some, but if the problem is actually that everything is behind the SMMU and we just don't support that at all, then...