7.1-STABLE crash

[Mikolaj Golub]

On Mon, 01 Jun 2009 10:05:46 +0400 Asmodean Dark wrote:

 AD> Hello,

 AD> We have cluster of FreeBSD VPN servers with running mpd3.18 and sometimes it crash:

Do you really need a such outdated version of mpd while mpd5 is available in
ports?

 AD> # uname -a
 AD> FreeBSD vpn 7.1-STABLE FreeBSD 7.1-STABLE #13: Wed Mar 18 14:53:13 YEKT 2009     root at vpn:/usr/src/sys/i386/compile/kconf  i386

 AD> # dmesg
 AD> Fatal trap 12: page fault while in kernel mode
 AD> cpuid = 1; apic id = 01
 AD> fault virtual address   = 0x8
 AD> fault code              = supervisor read, page not present
 AD> instruction pointer     = 0x20:0x806dcb88
 AD> stack pointer           = 0x28:0xe70775c4
 AD> frame pointer           = 0x28:0xe70775e8
 AD> code segment            = base 0x0, limit 0xfffff, type 0x1b
 AD>                         = DPL 0, pres 1, def32 1, gran 1
 AD> processor eflags        = interrupt enabled, resume, IOPL = 0
 AD> current process         = 22 (em0 taskq)
 AD> trap number             = 12

 AD> (kgdb) bt
 AD> #0  0x8063d6b0 in doadump ()
 AD> #1  0x8063dc44 in boot ()
 AD> #2  0x8063e0ca in panic ()
 AD> #3  0x807dab3d in trap_fatal ()
 AD> #4  0x807daeba in trap_pfault ()
 AD> #5  0x807db7bd in trap ()
 AD> #6  0x807c2a3b in calltrap ()
 AD> #7  0x806dcb88 in rn_match ()
 AD> #8  0x806ddc8a in rn_lookup ()
 AD> #9  0x8070e460 in ipfw_chk (args=0xe70175fc) at ../../../netinet/ip_fw2.c:1894
 AD> #10 0x80710c3d in ipfw_check_in (arg=0x0, m0=0xe7017700, ifp=0x91c5a800, dir=1, inp=0x0) at ../../../netinet/ip_fw_pfil.c:125
 AD> #11 0x806dc20f in pfil_run_hooks ()
 AD> #12 0x80713984 in ip_input (m=0x91954c00) at ../../../netinet/ip_input.c:416
 AD> #13 0x806ec0d9 in ng_iface_rcvdata ()
 AD> #14 0x806e9570 in ng_apply_item ()
 AD> #15 0x806e8569 in ng_snd_item ()
 AD> #16 0x806e9570 in ng_apply_item ()
 AD> #17 0x806e8569 in ng_snd_item ()
 AD> #18 0x806e9570 in ng_apply_item ()
 AD> #19 0x806e8569 in ng_snd_item ()
 AD> #20 0x806f16a7 in ng_ppp_proto_recv ()
 AD> #21 0x806f3ed2 in ng_ppp_rcvdata ()
 AD> #22 0x806e9570 in ng_apply_item ()
 AD> #23 0x806e8569 in ng_snd_item ()
 AD> #24 0x806e9570 in ng_apply_item ()
 AD> #25 0x806e8569 in ng_snd_item ()
 AD> #26 0x806ee3c3 in ng_ksocket_incoming2 ()
 AD> #27 0x806e969d in ng_apply_item ()
 AD> #28 0x806ea8aa in ngintr ()
 AD> #29 0x806dab72 in swi_net ()
 AD> #30 0x8061e265 in ithread_loop ()
 AD> #31 0x8061adf5 in fork_exit ()
 AD> #32 0x807c2ab0 in fork_trampoline ()

It looks like the kernel crashed when ipfw was looking for the packet's
src/dst in a table. The table might have been corrupted (being modified?)  at
that time. Do you have some automatic blocker or some other script that
periodically add/remove IPs in a ipfw table?

 AD> What can I do with it? Are additional info needed?

It is a bit strange for me that in the bt output the file source information
is displayed only for several functions. Do you have the kernel and all
modules built with the debugging symbols? If you don't I would recommend to
rebuild the kernel so I would be able to provide bt (from the next crash :-)
with all necessary info available. Do the bt output for other crashes looks
the same?

-- 
Mikolaj Golub