What is ENXIO – MSI allocation regression in :[Was Re: svn commit: r321714 - in head/sys/dev: mpr mps]

[Harry Schmalzbauer]

Am 01.08.2017 um 11:40 schrieb Harry Schmalzbauer:
> Bezüglich Scott Long's Nachricht vom 31.07.2017 18:56 (localtime):
>
> …
>>> I'd like to report one I hadn't expected:
>>>
>>> mps0: <Avago Technologies (LSI) SAS2008> port 0x4000-0x40ff mem 0xc3bc0000-0xc3bc3fff,0xc3b80000-0xc3bbffff irq 19 at device 0.0 on pci7
>>>
>>> mps0: Firmware: 20.00.04.00, Driver: 21.02.00.00-fbsd
>>> mps0: IOCCapabilities:
>>> 185c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,IR>
>>> mps0: Cannot allocate INTx interrupt
>>> mps0: mps_iocfacts_allocate failed to setup interrupts
>>> mps0: mps_attach IOC Facts based allocation failed with error 6
>>> panic: resource_list_release: resource entry is not busy
>>> cpuid = 6
>>> KDB: stack backtrace:
>>> #0 0xffffffff805e32d7 at kdb_backtrace+0x67
>>> #1 0xffffffff805a1d26 at vpanic+0x186
>>> #2 0xffffffff805a1b93 at panic+0x43
>>> #3 0xffffffff805d71c6 at resource_list_release+0x1c6
>>> #4 0xffffffff8040fef1 at mps_pci_free+0xe1
>>> #5 0xffffffff8040fa23 at mps_pci_attach+0x1b3
>>> #6 0xffffffff805d6594 at device_attach+0x3a4
>>> #7 0xffffffff805d774d at bus_generic_attach+0x3d
>>> #8 0xffffffff8044ac05 at pci_attach+0xd5
>>> #9 0xffffffff805d6594 at device_attach+0x3a4
>>> #10 0xffffffff805d774d at bus_generic_attach+0x3d
>>> #11 0xffffffff80364761 at acpi_pcib_pci_attach+0xa1
>>> #12 0xffffffff805d6594 at device_attach+0x3a4
>>> #13 0xffffffff805d774d at bus_generic_attach+0x3d
>>> #14 0xffffffff8044ac05 at pci_attach+0xd5
>>> #15 0xffffffff805d6594 at device_attach+0x3a4
>>> #16 0xffffffff805d774d at bus_generic_attach+0x3d
>>> #17 0xffffffff80363e4d at acpi_pcib_acpi_attach+0x42d
>>> Uptime: 1s
> …
>
>> Fixed in r321799, thanks for the report.
> Fix confiremd; merged together with r321733 (and 321737) to 11.1 and
> panic vanished.

Late in the 11.2 phase, I identified this commit as a regression for MSI 
(non-x) alloctaion.
I have an idea what probably causes the problem here (INTx allocation, 
although MSI (and MSI-x) capability):
disable_msix is not 0 (I need to disable MSI-x because of ESXi-passthru…).

Corresponding lines:
{
         device_t dev;
         int error, msgs;

         dev = sc->mps_dev;
         error = 0;
         msgs = 0;

         if ((sc->disable_msix == 0) &&
             ((msgs = pci_msix_count(dev)) >= MPS_MSI_COUNT))
                 error = mps_alloc_msix(sc, MPS_MSI_COUNT);
         if ((error != 0) && (sc->disable_msi == 0) &&
             ((msgs = pci_msi_count(dev)) >= MPS_MSI_COUNT))
                 error = mps_alloc_msi(sc, MPS_MSI_COUNT);
         if (error != 0)
                 msgs = 0;

         sc->msi_msgs = msgs;
         return (error);
}

Before r321714, error was assigned ENXIO, which, if != 0, could help 
make me understand the problem.
Unfortunately I have no idea what ENXIO means, where it's defined and 
most important, how to find the place where the declaration/definition 
happens.  Only joe and vi available here, any hints highly appreciated.

I can confirm that MSI allocation works with 
mps.ko_21.02.00.00-fbsd-r321415 with my ESXi-passthru-non_msi-x setup.
Although the dirver emits no message that an MSI was allocated, like 
toher drivers do.  That's a cosmetic one though.
But the MSI->INTx regression is a severe one for me, which I'd like to 
fix myself but I'm missing so many fundamental skills :-(

Thanks,

-harry