Unable to boot recent -stable with MSI/MSIX enabled

[Jack Vogel]

On Nov 15, 2007 7:17 AM, John Baldwin <jhb at freebsd.org> wrote:
> On Saturday 13 October 2007 11:59:40 am Alson van der Meulen wrote:
> > * Jack Vogel <jfvogel at gmail.com> [2007-10-13 05:19]:
> > > A suggestion, take the relevant files from my em driver and put
> > > them back into the kernel tree that was working on 10/1, it should
> > > be compatible. Then see if it breaks that kernel. Or if you'd prefer
> > > I can just email the tar ball for the Intel version of 6.6.6, you can
> > > disable the in-kernel em driver, and build the other and use that
> > > with the 10/1 kernel.
> >
>
> > Then I removed sys/dev/em and copied it from the 10/12 sources. I also
> > copied sys/conf/files, sys/conf/kern.pre.mk and sys/modules/em/Makelfile
> > from the 10/12 sources; this should be all of the 6.6.6 merge. Compiled
> > with same config, booted with MSI/MSIX enabled. Surprisingly, this
> > kernel behaved different than the 10/10 and 10/12 kernels. It booted OK
> > without any major errors, only a few watchdog timeouts and link down/ups
> > on em0. It was very slow though. Top showed 60% interrupt.
> >
> > vmstat -i:
> > interrupt                          total       rate
> > irq4: sio0                          3563          8
> > irq6: fdc0                             1          0
> > irq14: ata0                           58          0
> > irq16: fxp0                     32076072      79593
> > irq21: atapci1+                    24300         60
> > cpu0: timer                       793477       1968
> > Total                           32897471      81631
> >
> > There wasn't much traffic on fxp0 (only ssh, ping and ntp). According to
> > dmesg, em0 used the same IRQ as fxp0, except it should be using MSI:
> > em0: <Intel(R) PRO/1000 Network Connection Version - 6.6.6> port 0xdf00-0xdf1f mem 0xfdde0000-0xfddfffff,0xfddc0000-0xfdddffff irq 16 at device 0.0 on pci5
> > em0: Reserved 0x20000 bytes for rid 0x10 type 3 at 0xfdde0000
> > em0: attempting to allocate 1 MSI vectors (1 supported)
> > msi: routing MSI IRQ 256 to vector 56
> > em0: using IRQ 256 for MSI
> > em0: bpf attached
> > em0: Ethernet address: 00:15:17:19:59:e4
> > em0: [FAST]
> >
> > It appears that em0 still generates interrupts on irq16, even though it
> > should be using MSI.
>
> This was due to a bug with rman_set_rid() not getting used in 6-stable that
> broke the most recent MSI MFC.  The rman thing was fixed on 10/3, so MSI
> is not expected to work on 6-stable kernels between 8/15 and 10/3.
>
> Are you still having problems with em + MSI?

I got diverted with this S7000 bringup and haven't looked since that
last point in the thread. I am working in parallel on splitting the
E1000 driver and may be able to check this in a bit.

Jack