Re: What is ENXIO – MSI allocation regression in :[Was Re: svn commit: r321714 - in head/sys/dev: mpr mps]

[Harry Schmalzbauer]

Am 05.06.2018 um 19:54 schrieb Scott Long:
…
>>>> Late in the 11.2 phase, I identified this commit as a regression for MSI (non-x) alloctaion.
>>>> I have an idea what probably causes the problem here (INTx allocation, although MSI (and MSI-x) capability):
>>>> disable_msix is not 0 (I need to disable MSI-x because of ESXi-passthru…).
>>>>
>>>> Corresponding lines:
>>>> {
>>>>          device_t dev;
>>>>          int error, msgs;
>>>>
>>>>          dev = sc->mps_dev;
>>>>          error = 0;
>>>>          msgs = 0;
>>>>
>>>>          if ((sc->disable_msix == 0) &&
>>>>              ((msgs = pci_msix_count(dev)) >= MPS_MSI_COUNT))
>>>>                  error = mps_alloc_msix(sc, MPS_MSI_COUNT);
>>>>          if ((error != 0) && (sc->disable_msi == 0) &&
>>>>              ((msgs = pci_msi_count(dev)) >= MPS_MSI_COUNT))
>>>>                  error = mps_alloc_msi(sc, MPS_MSI_COUNT);
>>>>          if (error != 0)
>>>>                  msgs = 0;
>>>>
>>>>          sc->msi_msgs = msgs;
>>>>          return (error);
>>>> }
>>>>
>>>> Before r321714, error was assigned ENXIO, which, if != 0, could help make me understand the problem.
>>>> Unfortunately I have no idea what ENXIO means, where it's defined and most important, how to find the place where the declaration/definition happens.  Only joe and vi available here, any hints highly appreciated.
>>>>
>>>> I can confirm that MSI allocation works with mps.ko_21.02.00.00-fbsd-r321415 with my ESXi-passthru-non_msi-x setup.
>>>> Although the dirver emits no message that an MSI was allocated, like toher drivers do.  That's a cosmetic one though.
>>>> But the MSI->INTx regression is a severe one for me, which I'd like to fix myself but I'm missing so many fundamental skills :-(
>>>>
>>> Hi Harry,
>>> You are correct about the bug.  Please change the line at the top of the function that reads
>>> error = 0;
>>> to
>>> error = ENXIO;
>>> Let me know if that fixes the MSI problem for you.

…
>> BTW, does anybody have a link where I can get info about ENXIO?
> 
> ENXIO means that the device is not available.  I use it in the driver to signal when the hardware cannot be accessed.  The manual page for error codes is “man errno"

Oic, there's a man page :-)

Haven't had time to look into it, but since you confirmed that ENXIO!=0, 
I simply changed that and now mps(4) allocates MSI again in my setup.
For completeness my diff:
Index: src/sys/dev/mps/mps_pci.c
===================================================================
--- sys/dev/mps/mps_pci.c   (Revision 334948)
+++ sys/dev/mps/mps_pci.c   (Arbeitskopie)
@@ -244,7 +244,7 @@
         int error, msgs;

         dev = sc->mps_dev;
-       error = 0;
+       error = ENXIO;
         msgs = 0;

         if ((sc->disable_msix == 0) &&

Unfortunately my other real problem persists – iSCSI sessions lock up 
the machine (11.2-RC2).  No deadlock, since it will recover within some 
minutes, but otherwise a complete lock until iSCSI sessions time out, 
since no single ethernet/ip packet get's processed.

Unfortunately I'm very short on testing resources here and don't know 
how to trace ctld/whatelse to find the lock-circle.
So far I can only tell that it happens only with Server2016 iSCSI 
connections (using 4k block size).

Will open a different thread/PR as soon as I found out anything…

Thanks,

-harry

P.S.: I guess it's far too late to get that into 11.2?