em driver, 82574L chip, and possibly ASPM
Mike Tancsa
mike at sentex.net
Sat Dec 25 02:19:29 UTC 2010
On 12/24/2010 5:44 PM, Jan Koum wrote:
> hi Ivan and Mike,
>
> wanted to follow up and see if you found a solid long-term solution to this
> bug. we are still seeing this problem in our 8.2 environment with ASPM
> already disabled. here is what we have:
Hmmm,
With the latest version of the driver in RELENG_8 (its the same as in
HEAD) I havent seen the problem. However, I would only see it once per
week prior to that. The odd thing is that it would happen during a
slightly lower than normal backup load, but almost always at the same
time (early sunday AM). Not sure what would trigger it exactly. If it
happened again, I was going to enable port mirroring on the switchport
and capture the traffic, hoping some "special" pattern would enable the
issue.
Do you have IPMI enabled on the NIC ? I tried to turn it off on my MB,
but there is no clear way to do this. It 'seems' to be off, but not sure
if it really is. One thing I noticed was that when the NIC was hung, it
still was able to receive and process IPMI commands from an external host.
---Mike
>
> 1. motherboard is SuperMicro X8SIE-LN4F Intel Xeon:
>
> em0 at pci0:3:0:0: class=0x020000 card=0x040d15d9 chip=0x10d38086 rev=0x00
> hdr=0x00
> vendor = 'Intel Corporation'
> device = 'Intel 82574L Gigabit Ethernet Controller (82574L)'
> class = network
> subclass = ethernet
> em1 at pci0:4:0:0: class=0x020000 card=0x040d15d9 chip=0x10d38086 rev=0x00
> hdr=0x00
> vendor = 'Intel Corporation'
> device = 'Intel 82574L Gigabit Ethernet Controller (82574L)'
> class = network
> subclass = ethernet
> em2 at pci0:5:0:0: class=0x020000 card=0x040d15d9 chip=0x10d38086 rev=0x00
> hdr=0x00
> vendor = 'Intel Corporation'
> device = 'Intel 82574L Gigabit Ethernet Controller (82574L)'
> class = network
> subclass = ethernet
> em3 at pci0:6:0:0: class=0x020000 card=0x040d15d9 chip=0x10d38086 rev=0x00
> hdr=0x00
> vendor = 'Intel Corporation'
> device = 'Intel 82574L Gigabit Ethernet Controller (82574L)'
> class = network
> subclass = ethernet
>
> 2. ASPM is already disabled in the BIOS
>
> 3. when em1 interface locks up, sysctl debug says:
>
> Interface is NOT RUNNING
> and INACTIVE
> em1: hw tdh = 0, hw tdt = 0
> em1: hw rdh = 0, hw rdt = 0
> em1: Tx Queue Status = 0
> em1: TX descriptors avail = 110
> em1: Tx Descriptors avail failure = 319
> em1: RX discarded packets = 0
> em1: RX Next to Check = 80
> em1: RX Next to Refresh = 80
>
> 4. doing "ifconfig em1 down; sleep1; ifconfig em1 up" resolves the issue and
> removes OACTIVE flag from em1.
>
> 5. we are running 8.2-PRERELEASE from December 19th:
> % grep '$FreeBSD' /usr/src/sys/dev/e1000/if_em.c
> /*$FreeBSD: src/sys/dev/e1000/if_em.c,v 1.21.2.18 2010/12/14 19:59:39 jfv
> Exp $*/
>
> dmesg output is:
>
> em1: <Intel(R) PRO/1000 Network Connection 7.1.8> port 0xcc00-0xcc1f mem
> 0xfb4e0000-0xfb4fffff,0xfb4dc000-0xfb4dffff irq 17 at device 0.0 on pci4
> em1: Reserved 0x20000 bytes for rid 0x10 type 3 at 0xfb4e0000
> em1: Reserved 0x4000 bytes for rid 0x1c type 3 at 0xfb4dc000
> em1: attempting to allocate 3 MSI-X vectors (5 supported)
> msi: routing MSI-X IRQ 259 to local APIC 0 vector 53
> msi: routing MSI-X IRQ 260 to local APIC 0 vector 54
> msi: routing MSI-X IRQ 261 to local APIC 0 vector 55
> em1: using IRQs 259-261 for MSI-X
> em1: Using MSIX interrupts with 3 vectors
> em1: [MPSAFE]
> em1: [ITHREAD]
> em1: [MPSAFE]
> em1: [ITHREAD]
> em1: [MPSAFE]
> em1: [ITHREAD]
> em1: bpf attached
> em1: Ethernet address: 00:25:90:0e:25:e9
>
> aside from running cronjob every minute to check for dead interface and
> reset it, is there anything else we can try?
>
> thanks.
>
>
> On Tue, Nov 23, 2010 at 10:36 AM, Jack Vogel <jfvogel at gmail.com> wrote:
>
>> 82574 is supposed to be em, not igb :) Its always had this kind of
>> 'in-between'
>> status, it was targeted as a 'client' or consumer part, but it has MSIX
>> which
>> make it almost like 8257[56].
>>
>> Mike, there are some further 82574 changes to shared code that I'm looking
>> into today.
>>
>> Jack
>>
>>
>> On Tue, Nov 23, 2010 at 10:17 AM, Mike Tancsa <mike at sentex.net> wrote:
>>
>>> On 11/23/2010 12:39 PM, Sean Bruno wrote:
>>>> On Tue, 2010-11-23 at 04:47 -0800, Ivan Voras wrote:
>>>>> It looks like I'm unfortunate enough to have to deploy on a machine
>>>>> which has the 82574L Intel NIC chip on a Supermicro X8SIE-F board,
>> which
>>>> igb0 at pci0:5:0:0: class=0x020000 card=0x8975152d chip=0x10c98086
>>>
>>> Strange, the 82574 attaches as em for me, not igb
>>>
>>> em1 at pci0:10:0:0: class=0x020000 card=0x34ec8086 chip=0x10d38086
>>> rev=0x00 hdr=0x00
>>> vendor = 'Intel Corporation'
>>> device = 'Intel 82574L Gigabit Ethernet Controller (82574L)'
>>> class = network
>>> subclass = ethernet
>>> cap 01[c8] = powerspec 2 supports D0 D3 current D0
>>> cap 05[d0] = MSI supports 1 message, 64 bit enabled with 1 message
>>> cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1)
>>> cap 11[a0] = MSI-X supports 5 messages in map 0x1c
>>> ecap 0001[100] = AER 1 0 fatal 0 non-fatal 0 corrected
>>> ecap 0003[140] = Serial 1 001517ffffed68a4
>>>
>>> Normally, its msix, but I had disabled that hoping it would fix the
>> problem
>>>
>>> em1: <Intel(R) PRO/1000 Network Connection 7.1.7> port 0x2000-0x201f mem
>>> 0xb4100000-0xb411ffff,0xb4120000-0xb4123fff irq 16 at dev
>>> ice 0.0 on pci10
>>> em1: Using an MSI interrupt
>>> em1: [FILTER]
>>> em1: Ethernet address: 00:15:17:ed:68:a4
>>>
>>>
>>> ---Mike
>>> _______________________________________________
>>> freebsd-net at freebsd.org mailing list
>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net
>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
>>>
>> _______________________________________________
>> freebsd-hardware at freebsd.org mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-hardware
>> To unsubscribe, send any mail to "freebsd-hardware-unsubscribe at freebsd.org
>> "
>>
>
More information about the freebsd-hardware
mailing list