em driver, 82574L chip, and possibly ASPM

[Jan Koum]

hi Ivan and Mike,

wanted to follow up and see if you found a solid long-term solution to this
bug. we are still seeing this problem in our 8.2 environment with ASPM
already disabled.  here is what we have:

1. motherboard is SuperMicro X8SIE-LN4F Intel Xeon:

em0 at pci0:3:0:0: class=0x020000 card=0x040d15d9 chip=0x10d38086 rev=0x00
hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'Intel 82574L Gigabit Ethernet Controller (82574L)'
    class      = network
    subclass   = ethernet
em1 at pci0:4:0:0: class=0x020000 card=0x040d15d9 chip=0x10d38086 rev=0x00
hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'Intel 82574L Gigabit Ethernet Controller (82574L)'
    class      = network
    subclass   = ethernet
em2 at pci0:5:0:0: class=0x020000 card=0x040d15d9 chip=0x10d38086 rev=0x00
hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'Intel 82574L Gigabit Ethernet Controller (82574L)'
    class      = network
    subclass   = ethernet
em3 at pci0:6:0:0: class=0x020000 card=0x040d15d9 chip=0x10d38086 rev=0x00
hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'Intel 82574L Gigabit Ethernet Controller (82574L)'
    class      = network
    subclass   = ethernet

2. ASPM is already disabled in the BIOS

3. when em1 interface locks up, sysctl debug says:

Interface is NOT RUNNING
and INACTIVE
em1: hw tdh = 0, hw tdt = 0
em1: hw rdh = 0, hw rdt = 0
em1: Tx Queue Status = 0
em1: TX descriptors avail = 110
em1: Tx Descriptors avail failure = 319
em1: RX discarded packets = 0
em1: RX Next to Check = 80
em1: RX Next to Refresh = 80

4. doing "ifconfig em1 down; sleep1; ifconfig em1 up" resolves the issue and
removes OACTIVE flag from em1.

5. we are running 8.2-PRERELEASE from December 19th:
% grep '$FreeBSD' /usr/src/sys/dev/e1000/if_em.c
/*$FreeBSD: src/sys/dev/e1000/if_em.c,v 1.21.2.18 2010/12/14 19:59:39 jfv
Exp $*/

dmesg output is:

em1: <Intel(R) PRO/1000 Network Connection 7.1.8> port 0xcc00-0xcc1f mem
0xfb4e0000-0xfb4fffff,0xfb4dc000-0xfb4dffff irq 17 at device 0.0 on pci4
em1: Reserved 0x20000 bytes for rid 0x10 type 3 at 0xfb4e0000
em1: Reserved 0x4000 bytes for rid 0x1c type 3 at 0xfb4dc000
em1: attempting to allocate 3 MSI-X vectors (5 supported)
msi: routing MSI-X IRQ 259 to local APIC 0 vector 53
msi: routing MSI-X IRQ 260 to local APIC 0 vector 54
msi: routing MSI-X IRQ 261 to local APIC 0 vector 55
em1: using IRQs 259-261 for MSI-X
em1: Using MSIX interrupts with 3 vectors
em1: [MPSAFE]
em1: [ITHREAD]
em1: [MPSAFE]
em1: [ITHREAD]
em1: [MPSAFE]
em1: [ITHREAD]
em1: bpf attached
em1: Ethernet address: 00:25:90:0e:25:e9

aside from running cronjob every minute to check for dead interface and
reset it, is there anything else we can try?

thanks.


On Tue, Nov 23, 2010 at 10:36 AM, Jack Vogel <jfvogel at gmail.com> wrote:

> 82574 is supposed to be em, not igb :)  Its always had this kind of
> 'in-between'
> status, it was targeted as a 'client' or consumer part, but it has MSIX
> which
> make it almost like 8257[56].
>
> Mike, there are some further 82574 changes to shared code that I'm looking
> into today.
>
> Jack
>
>
> On Tue, Nov 23, 2010 at 10:17 AM, Mike Tancsa <mike at sentex.net> wrote:
>
> > On 11/23/2010 12:39 PM, Sean Bruno wrote:
> > > On Tue, 2010-11-23 at 04:47 -0800, Ivan Voras wrote:
> > >> It looks like I'm unfortunate enough to have to deploy on a machine
> > >> which has the 82574L Intel NIC chip on a Supermicro X8SIE-F board,
> which
> > > igb0 at pci0:5:0:0:        class=0x020000 card=0x8975152d chip=0x10c98086
> >
> > Strange, the 82574 attaches as em for me, not igb
> >
> > em1 at pci0:10:0:0:        class=0x020000 card=0x34ec8086 chip=0x10d38086
> > rev=0x00 hdr=0x00
> >    vendor     = 'Intel Corporation'
> >    device     = 'Intel 82574L Gigabit Ethernet Controller (82574L)'
> >    class      = network
> >    subclass   = ethernet
> >    cap 01[c8] = powerspec 2  supports D0 D3  current D0
> >    cap 05[d0] = MSI supports 1 message, 64 bit enabled with 1 message
> >    cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1)
> >    cap 11[a0] = MSI-X supports 5 messages in map 0x1c
> > ecap 0001[100] = AER 1 0 fatal 0 non-fatal 0 corrected
> > ecap 0003[140] = Serial 1 001517ffffed68a4
> >
> > Normally, its msix, but I had disabled that hoping it would fix the
> problem
> >
> > em1: <Intel(R) PRO/1000 Network Connection 7.1.7> port 0x2000-0x201f mem
> > 0xb4100000-0xb411ffff,0xb4120000-0xb4123fff irq 16 at dev
> > ice 0.0 on pci10
> > em1: Using an MSI interrupt
> > em1: [FILTER]
> > em1: Ethernet address: 00:15:17:ed:68:a4
> >
> >
> >        ---Mike
> > _______________________________________________
> > freebsd-net at freebsd.org mailing list
> > http://lists.freebsd.org/mailman/listinfo/freebsd-net
> > To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
> >
> _______________________________________________
> freebsd-hardware at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-hardware
> To unsubscribe, send any mail to "freebsd-hardware-unsubscribe at freebsd.org
> "
>