kern/163117: sporadic boot-time panics on 9.0-RC2 and 9.0-RC3 w/ certain hardware

[Mike Andrews]

>Number:         163117
>Category:       kern
>Synopsis:       sporadic boot-time panics on 9.0-RC2 and 9.0-RC3 w/ certain hardware
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Thu Dec 08 02:50:11 UTC 2011
>Closed-Date:
>Last-Modified:
>Originator:     Mike Andrews
>Release:        FreeBSD 9.0-RC3 amd64
>Organization:
>Environment:
System: FreeBSD brandy.fark.com 9.0-RC3 FreeBSD 9.0-RC3 #32 r228247: Sat Dec  3 18:57:23 EST 2011     mandrews at beer.int.fark.com:/usr/obj/usr/src/sys/FARK64  amd64

	Supermicro X8STi-F

>Description:
	
*Sometimes* when booting 9.0-RC2 or 9.0-RC3 on *some* of my machines,
I'll get one of the following two panics during multiuser startup,
usually while running the /usr/local/etc/rc.d scripts. (The instruction
pointer is always exactly one of these two, and they look fairly
related.) If after two or three reboots it manages to not panic, the
system will run perfectly stable. 

Some discussion here: http://www.mail-archive.com/freebsd-stable@freebsd.org/msg118075.html

Console logs and kernel configs here: http://www.bit0.com/tmp/panic90rc3fark.tar.gz

I've run this kernel on the following boards, all of which have em(4) NICs of varying vintage:

Supermicro X8STi-F
  em0 at pci0:1:0:0:        class=0x020000 card=0x10d315d9 chip=0x10d38086 rev=0x00 hdr=0x00
  em1 at pci0:2:0:0:        class=0x020000 card=0x10d315d9 chip=0x10d38086 rev=0x00 hdr=0x00
Supermicro X8SIE-F
  em0 at pci0:3:0:0:        class=0x020000 card=0x040d15d9 chip=0x10d38086 rev=0x00 hdr=0x00
  em1 at pci0:4:0:0:        class=0x020000 card=0x040d15d9 chip=0x10d38086 rev=0x00 hdr=0x00
Supermicro X7SBE
  em0 at pci0:13:0:0:       class=0x020000 card=0x108c15d9 chip=0x108c8086 rev=0x03 hdr=0x00
  em1 at pci0:15:0:0:       class=0x020000 card=0x109a15d9 chip=0x109a8086 rev=0x00 hdr=0x00
Supermicro PDSMi+ and PDSMA+
  em0 at pci0:13:0:0:       class=0x020000 card=0x108c15d9 chip=0x108c8086 rev=0x03 hdr=0x00
  em1 at pci0:14:0:0:       class=0x020000 card=0x109a15d9 chip=0x109a8086 rev=0x00 hdr=0x00
Intel S3000AHLX
  em0 at pci0:4:0:0:        class=0x020000 card=0x348d8086 chip=0x108c8086 rev=0x03 hdr=0x00
  em1 at pci0:5:5:0:        class=0x020000 card=0x348d8086 chip=0x10768086 rev=0x05 hdr=0x00
Intel S3000AHV
  em0 at pci0:3:0:0:        class=0x020000 card=0x34898086 chip=0x108c8086 rev=0x03 hdr=0x00
Supermicro X7SPA-HF
  em0 at pci0:2:0:0:        class=0x020000 card=0x060a15d9 chip=0x10d38086 rev=0x00 hdr=0x00
  em1 at pci0:3:0:0:        class=0x020000 card=0x060a15d9 chip=0x10d38086 rev=0x00 hdr=0x00

...and as far as I can tell, only the X8STi-F has the issue.  All the others seem
to boot up fine every time.  It's just the first that randomly panics (or doesn't)
on boot.

Supermicro X7SLA-H with re(4) NICs works fine too


-----
First 9.0-RC2 panic (note em0 warning before it):
-----
em0: discard frame w/o packet header


Fatal trap 9: general protection fault while in kernel mode
cpuid = 0; apic id = 00
instruction pointer     = 0x20:0xffffffff805e4fc5
stack pointer           = 0x28:0xffffff80003299e0
frame pointer           = 0x28:0xffffff8000329a00
code segment            = base 0x0, limit 0xfffff, type 0x1b
                         = DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags        = interrupt enabled, resume, IOPL = 0
current process         = 12 (irq256: em0:rx 0)
trap number             = 9
panic: general protection fault
cpuid = 0
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2a
kdb_backtrace() at kdb_backtrace+0x37
panic() at panic+0x187
trap_fatal() at trap_fatal+0x290
trap() at trap+0x10a
calltrap() at calltrap+0x8
--- trap 0x9, rip = 0xffffffff805e4fc5, rsp = 0xffffff80003299e0, rbp = 0xffffff8000329a00 ---
m_freem() at m_freem+0x25
ether_nh_input() at ether_nh_input+0x82
netisr_dispatch_src() at netisr_dispatch_src+0x20b
em_rxeof() at em_rxeof+0x1ca
em_msix_rx() at em_msix_rx+0x24
intr_event_execute_handlers() at intr_event_execute_handlers+0x104
ithread_loop() at ithread_loop+0xa4
fork_exit() at fork_exit+0x11f
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff8000329d00, rbp = 0 ---
Uptime: 49s
Dumping 679 out of 12263 MB:

-----
Second 9.0-RC2 panic (no em0 discard warning this time):
-----

Fatal trap 9: general protection fault while in kernel mode
cpuid = 0; apic id = 00
instruction pointer     = 0x20:0xffffffff8063c0e4
stack pointer           = 0x28:0xffffff8000329a00
frame pointer           = 0x28:0xffffff8000329a40
code segment            = base 0x0, limit 0xfffff, type 0x1b
                         = DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags        = interrupt enabled, resume, IOPL = 0
current process         = 12 (irq256: em0:rx 0)
trap number             = 9
panic: general protection fault
cpuid = 0
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2a
kdb_backtrace() at kdb_backtrace+0x37
panic() at panic+0x187
trap_fatal() at trap_fatal+0x290
trap() at trap+0x10a
calltrap() at calltrap+0x8
--- trap 0x9, rip = 0xffffffff8063c0e4, rsp = 0xffffff8000329a00, rbp = 0xffffff8000329a40 ---
ether_nh_input() at ether_nh_input+0x94
netisr_dispatch_src() at netisr_dispatch_src+0x20b
em_rxeof() at em_rxeof+0x1ca
em_msix_rx() at em_msix_rx+0x24
intr_event_execute_handlers() at intr_event_execute_handlers+0x104
ithread_loop() at ithread_loop+0xa4
fork_exit() at fork_exit+0x11f
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff8000329d00, rbp = 0 ---
Uptime: 46s
Dumping 657 out of 12263 MB:..3%

>How-To-Repeat:

	Boot a releng/9.0 kernel on a Supermicro X8STi-F

>Fix:

	Use a different motherboard?



>Release-Note:
>Audit-Trail:
>Unformatted: