FreeBSD on Layerscape/QorIQ LX2160X

[greg at unrelenting.technology]

May 23, 2020 4:46 AM, "myfreeweb" <greg at unrelenting.technology> wrote:

> You can see that in the gist: "NVME polled command failed to complete within 1s"
> 
> Same as before (with straight ECAM).
> 
> Looks like straight ECAM *is* supposed to work, after all (with limitations on bifurcation or
> something, I've heard). The code I'm porting from NetBSD does use that same ECAM space when
> touching non-0 busses.
> 
> Whoops, I might've been working on increasing features instead of fixing bugs.. :D
> 
> It's odd that what's failing is just polling I/O with the devices, and after successful reads. Does
> that smell like cache issues or something?

I was wondering about NetBSD's usage of _ARM_BUS_SPACE_MAP_STRONGLY_ORDERED,
but that's just their flag for using Device_nGnRnE instead of Device_nGnRE.
We always use Device_nGnRnE.
(Probably leaving some performance on the table for non-PCIe devices..)


Also, comparing the dsdt for the LX2160A vs. Armada8k:

https://github.com/SolidRun/edk2-platforms/blob/eec706c2d693be0b3793d9180e7d1a4813a526cf/Silicon/NXP/LX2160A/AcpiTables/Dsdt/Pci.asl
https://github.com/tianocore/edk2-platforms/blob/7a4035e9efd890215c88ccbc645b7e9e3671779f/Silicon/Marvell/Armada7k8k/AcpiTables/Armada80x0McBin/Dsdt.asl#L308

LX2160A only offers bus numbers and 64-bit MMIO, while Armada8k also offers 32-bit MMIO and translated I/O ports.
So some of the pci_host_generic_core_alloc_resource FAILs are expected I guess.

Interestingly, NetBSD has never complained about this, see dmesgs:

early: http://www.netbsd.org/~jmcneill/lx2k.txt
after adding special support: https://dmesgd.nycbug.org/index.cgi?do=view&id=5335

and the config space support seems to have fixed everything for them (??)


I also looked more closely at mps, and actually "just polling I/O" does work. We see:
mps0: IOCCapabilities: 1285c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,EventReplay,HostDisc>
and that means mps_request_sync (called by mps_get_iocfacts) has worked.

And the command completion actually seems to come from interrupts.

I've heard something about MSI(X) not working for the Linux people (not NetBSD though!),
so I guess the next thing to try would be booting with hw.pci.enable_msi=0 hw.pci.enable_msi=1