FreeBSD on Layerscape/QorIQ LX2160X

[Dan Kotowski]

> > > Oops, I've been adding some debug prints to gicv2 not gicv3.
> > > One thing I noticed is the interesting irq number the nvme admin queue gets..
> > > it's the same as gic_nirqs.
> > > gic0: SPIs: 288, IDs: 65535
> > > nvme0: attempting to allocate 17 MSI-X vectors (33 supported)
> > > nvme0: using IRQs 21-37 for MSI-X
> > > acpi0: allocating via sysres: res 0xfffffd0000726280, start 21 + count 1 - 1 =? end 21
> > > intr_setup_irq(): irq 288 add handler error 0 on nvme0
> > > maybe that's just like.. the first ITS handled interrupt?
> > > (funnily enough, NetBSD lists MSIs as IRQs starting from 8192)
> > > More SATA debugging: https://send.firefox.com/download/9de5357a2e58edd9/#s9ZaU_k2NHlO-tLdyGk5iA
> >
> > https://gist.github.com/agrajag9/1aaf18f48ee883b917907dd417803711
> > We have SATA!
>
> Wow that was silly.
>
> FreeBSD did not support multiple interrupts in an ACPI device, at all.
> Since forever, nobody else has needed this, apparently.
> It's really stupid that the error was returned this quietly >_<
>
> In the build below, it would try to attach all interrupts, not just the first one.
> If that succeeds, I'll post that as a patch.
>
> > NVMe and mps still cause hangs, but progress is progress!
>
> Well I didn't say I've tried to actually fix PCIe interrupts yet,
> but I did add some debug logging that seems to be useful.
> Namely, IORT stuff. IORT is the ACPI table describing how interrupts flow.
>
> Aaaand this is concerning:
>
> iort: Found PCI RC node with segment and device ID. its? 1 smmu3? 0 smmu? 1 | nxtid 6400 rid 256
> iort: entry lookup fail
>
> Wait a second, SMMU, that's not what IORT says the PCI nodes output to!
>
> https://github.com/SolidRun/edk2-platforms/blob/eec706c2d693be0b3793d9180e7d1a4813a526cf/Silicon/NXP
> LX2160A/AcpiTables/Iort.aslc
>
> .PciRcNode[0] = { .PciRcIdMapping = { ...
> .OutputReference = OFFSET_OF (NXP_EFI_ACPI_6_0_IO_REMAPPING_TABLE, ItsNode), },
>
> Looks like our IORT parsing might be busted!
>
> The new build should output more info and force usage of the ITS node.
> So if that was the issue, and I did the forcing correctly, you should have PCIe interrupts working now :)
>
> (btw: NetBSD for now doesn't care about OutputReference at all, lol)
>
> https://send.firefox.com/download/d655b27c2bd73021/#O9tR6di9HbS8FHp0isgFkA

https://gist.github.com/agrajag9/749f504dcac741e902f87c2ea03ae9ac

>From jnettlet:

BEGIN QUOTE
the V1 silicon has a SATA errata.  I have partially worked around the errata moving some configuration into the firmware, but I still need to sort out the remaining bits.  Sometimes just unplugging and replugging your SATA cable will help to reseat the connector and help.   Also it is very sensitive to marginal SATA cables.
END QUOTE

I played around a bit and could create and destroy GPTs, make filesystems, read and write to said filesystems, and hot-plug even seems to work. But I'm not sure if jon's comment really explains the AHCI error we're still seeing:

```
(aprobe3:ahcich3:0:15:0): NOP FLUSHQUEUE. ACB: 00 00 00 00 00 00 00 00 00 00 00 00
(aprobe3:ahcich3:0:15:0): CAM status: Command timeout
(aprobe3:ahcich3:0:15:0): Error 5, Retries exhausted
```