Crashes on X7SPE-HF with em

[Pyun YongHyeon]

On Fri, Aug 27, 2010 at 12:28:14AM +0200, Philipp Wuensche wrote:
> Jeremy Chadwick wrote:
> > On Thu, Aug 26, 2010 at 11:56:48PM +0200, Philipp Wuensche wrote:
> >> Jeremy Chadwick wrote:
> >>> CC'ing Jack Vogel of Intel and Yong-Hyeon PYUN who might have some
> >>> ideas.  OP's backtrace is here:
> >>>
> >>> http://lists.freebsd.org/pipermail/freebsd-stable/2010-August/058425.html
> >>>
> >>> Philipp, can you please provide the following output?
> >>>
> >>> * dmesg | egrep 'em[0-9]'
> >> em0: <Intel(R) PRO/1000 Network Connection 7.0.5> port 0xdc00-0xdc1f mem
> >> 0xfe9e0000-0xfe9fffff,0xfe9dc000-0xfe9dffff irq 16 at device 0.0 on pci2
> >> em0: Using MSI interrupt
> >> em0: [FILTER]
> >> em0: Ethernet address: 00:25:90:04:6e:fa
> >> em1: <Intel(R) PRO/1000 Network Connection 7.0.5> port 0xec00-0xec1f mem
> >> 0xfeae0000-0xfeafffff,0xfeadc000-0xfeadffff irq 17 at device 0.0 on pci3
> >> em1: Using MSI interrupt
> >> em1: [FILTER]
> >> em1: Ethernet address: 00:25:90:04:6e:fb
> >>
> >>> * uname -a         (you can XXX out the machine name if need be)
> >> FreeBSD XXX 8.1-STABLE FreeBSD 8.1-STABLE #2: Wed Aug 25 10:38:50 CEST
> >> 2010     root at XXX:/usr/obj/usr/src/sys/XXX  amd64
> >>
> >> Date of source is Aug 17 14:09 CEST 2010. It happend with 8.1-RELEASE
> >> too, I can go back to RELEASE or any SVN revision you would like, if it
> >> is helping in any way.
> >>
> >> Kernel-config:
> >>
> >> include         GENERIC
> >>
> >> ident           XXX
> >>
> >> options         IPSEC
> >>
> >> options	DEVICE_POLLING
> >> options     ACCEPT_FILTER_HTTP
> >>
> >> options ALTQ
> >>
> >> options ALTQ_CBQ
> >> options ALTQ_RED
> >> options ALTQ_RIO
> >> options ALTQ_HFSC
> >> options ALTQ_PRIQ
> >>
> >> device		crypto
> >> device		enc
> >>
> >>
> >>> * pciconf -lvc     (only include the em(4) items please)
> >> em0 at pci0:2:0:0:	class=0x020000 card=0x060a15d9 chip=0x10d38086 rev=0x00
> >> hdr=0x00
> >>     vendor     = 'Intel Corporation'
> >>     device     = 'Intel 82574L Gigabit Ethernet Controller (82574L)'
> >>     class      = network
> >>     subclass   = ethernet
> >>     cap 01[c8] = powerspec 2  supports D0 D3  current D0
> >>     cap 05[d0] = MSI supports 1 message, 64 bit enabled with 1 message
> >>     cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1)
> >>     cap 11[a0] = MSI-X supports 5 messages in map 0x1c
> >> em1 at pci0:3:0:0:	class=0x020000 card=0x060a15d9 chip=0x10d38086 rev=0x00
> >> hdr=0x00
> >>     vendor     = 'Intel Corporation'
> >>     device     = 'Intel 82574L Gigabit Ethernet Controller (82574L)'
> >>     class      = network
> >>     subclass   = ethernet
> >>     cap 01[c8] = powerspec 2  supports D0 D3  current D0
> >>     cap 05[d0] = MSI supports 1 message, 64 bit enabled with 1 message
> >>     cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1)
> >>     cap 11[a0] = MSI-X supports 5 messages in map 0x1c
> >>
> >>> * vmstat -i
> >> interrupt                          total       rate
> >> irq1: atkbd0                           9          0
> >> cpu0: timer                     36544552       1994
> >> irq256: em0                         3801          0
> >> irq257: em1                     32963909       1799
> >> irq258: ahci0                     175662          9
> >> cpu1: timer                     36543525       1994
> >> cpu2: timer                     36543525       1994
> >> cpu3: timer                     36543525       1994
> >> Total                          179318508       9786
> >>
> >> There is an shared IPMI interface on em0, but the interface is not used
> >> by FreeBSD. em1 is used by four VLANs. Polling is only in the
> >> Kernelconfig, not activated on the devices.
> > 
> > So much complexity here.  Tracking this down might be difficult.
> > 
> > One thing that does concern me is the interrupt rate for em1.  Jack et
> > al, is this normal?  I don't see this behaviour on my 8.x systems with
> > em(4) driver 7.0.5, but my systems all use 82573E and 82573L, and don't
> > have MSI-X support.
> 
> This is with the current settings, RXCSUM,TXCSUM and TSO disabled.
> Uptime is now only 5h:44m. It crashes about 2 to 3 times a day. I
> haven't found a way to trigger this, so I can only wait for it happen again.
> 

I'm not sure but it does not look like em(4) issue. Of course em(4)
could free/poke mbuf which was already passed to upper stack but
this may happen only when it has severe bug and I didn't see such
code yet.
It seems you're using ipsec, is it doable for you to disable ipsec
to narrow down the issue?