kern/167500: Kernel panics in em driver

[J David]

>Number:         167500
>Category:       kern
>Synopsis:       Kernel panics in em driver
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Tue May 01 20:50:10 UTC 2012
>Closed-Date:
>Last-Modified:
>Originator:     J David
>Release:        8.3-STABLE r234781
>Organization:
>Environment:
FreeBSD r1 8.3-STABLE FreeBSD 8.3-STABLE #5 r234781: Sun Apr 29 21:28:10 UTC 2012     root at r1:/data/r1/freebsd/obj/data/r1/freebsd/8-STABLE/sys/ROUTER64  amd64

>Description:

We experience frequent panics under heavy network load with FreeBSD 8.3 that appear to be related to the em driver:


Fatal trap 9: general protection fault while in kernel mode
cpuid = 1; apic id = 01
instruction pointer     = 0x20:0xffffffff806cd1d5
stack pointer           = 0x28:0xffffff80000f1450
frame pointer           = 0x28:0xffffff80000f1470
code segment            = base 0x0, limit 0xfffff, type 0x1b
                       = DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags        = interrupt enabled, resume, IOPL = 0
current process         = 0 (em1 que)
trap number             = 9
panic: general protection fault
cpuid = 1
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2a
kdb_backtrace() at kdb_backtrace+0x37
panic() at panic+0x183
trap_fatal() at trap_fatal+0x290
trap() at trap+0x105
calltrap() at calltrap+0x8
--- trap 0x9, rip = 0xffffffff806cd1d5, rsp = 0xffffff80000f1450, rbp
= 0xffffff80000f1470 ---
m_freem() at m_freem+0x25
em_txeof() at em_txeof+0x164
em_start_locked() at em_start_locked+0x718
em_start() at em_start+0x5c
if_transmit() at if_transmit+0xea
lagg_start() at lagg_start+0x17b
if_transmit() at if_transmit+0xea
vlan_transmit() at vlan_transmit+0x11d
ether_output_frame() at ether_output_frame+0x33
ether_output() at ether_output+0x518
ip_fastforward() at ip_fastforward+0x4a5
ether_demux() at ether_demux+0x198
ether_input() at ether_input+0x197
ether_demux() at ether_demux+0x6f
ether_input() at ether_input+0x197
em_rxeof() at em_rxeof+0x1c7
em_handle_que() at em_handle_que+0x52
taskqueue_run_locked() at taskqueue_run_locked+0x85
taskqueue_thread_loop() at taskqueue_thread_loop+0x4e
fork_exit() at fork_exit+0x11f
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff80000f1d00, rbp = 0 ---
Uptime: 9h55m2s
Cannot dump. Device not defined or unavailable.
Automatic reboot in 15 seconds - press a key on the console to abort
--> Press a key on the console to reboot,
em2: discard frame w/o packet header

Fatal trap 9: general protection fault while in kernel mode
cpuid = 1; apic id = 01
instruction pointer     = 0x20:0xffffffff806cd1d5
stack pointer           = 0x28:0xffffff800011ba50
frame pointer           = 0x28:0xffffff800011ba70
code segment            = base 0x0, limit 0xfffff, type 0x1b
                       = DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags        = interrupt enabled, resume, IOPL = 0
current process         = 0 (em2 que)
trap number             = 9
panic: general protection fault
cpuid = 1


This machine is a pf firewall under a pretty heavy traffic load.  The
line "em2: discard frame w/o packet header" seems to be the most
informative diagnostic, because all that interface does is pfsync to
the another machine; it's under very light load compared to em0, em1,
and em3.

>How-To-Repeat:

1) Configure machine as FreeBSD OpenBGPd router/ pf firewall.
2) Punish it.
3) Wait.

>Fix:


>Release-Note:
>Audit-Trail:
>Unformatted: