kernel trap 12 with interrupts disabled [bge0 on 7.2R]

Martin nakal at web.de
Thu May 14 17:10:31 UTC 2009


Am Thu, 14 May 2009 09:16:40 -0400
schrieb John Baldwin <jhb at freebsd.org>:

> On Thursday 14 May 2009 7:47:23 am Martin Sugioarto wrote:
> [...]
> > kernel trap 12 with interrupts disabled
> > 
> > 
> > Fatal trap 12: page fault while in kernel mode
> > cpuid = 0; apic id = 0
> > fault virtual address = 0x80000000000
> 
> Given that that is a single bit set, it could possibly be due to bad
> RAM.

This is the second panic output that appeared on the screen. I could not read
the first lines of the first panic. The last ones looked similar
(same trap/process etc).

> Does your kernel have debug symbols?

This is GENERIC kernel configuration. The kernel was totally frozen. I could
not type anything. I just noticed, I've got a vmcore.0 of the crash.

I can see some other panic output when loading the kernel in kgdb:

Unread portion of the kernel message buffer:


Fatal trap 9: general protection fault while in kernel mode
cpuid = 2; apic id = 02
instruction pointer	= 0x8:0xffffffff805bbc66
stack pointer	        = 0x10:0xffffffff51e2e410
frame pointer	        = 0x10:0xffffffff51e2e4c0
code segment		= base 0x0, limit 0xfffff, type 0x1b
			= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags	= interrupt enabled, resume, IOPL = 0
current process		= 1311 (nfsiod 0)
trap number		= 9
panic: general protection fault
cpuid = 2
Uptime: 1h5m39s
Physical memory: 8179 MB
Dumping 479 MB: 464 448 432 416 400 384 368 352 336 320 304 288 272 256 240 224 208 192 176 160 144 128 112 96 80 64 48 32 16

Reading symbols from /boot/kernel/geom_journal.ko...Reading symbols from /boot/kernel/geom_journal.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/geom_journal.ko
Reading symbols from /boot/kernel/nullfs.ko...Reading symbols from /boot/kernel/nullfs.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/nullfs.ko
Reading symbols from /boot/kernel/pflog.ko...Reading symbols from /boot/kernel/pflog.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/pflog.ko
Reading symbols from /boot/kernel/pf.ko...Reading symbols from /boot/kernel/pf.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/pf.ko
#0  doadump () at pcpu.h:195
195		__asm __volatile("movq %%gs:0,%0" : "=r" (td));

Here the backtrace:
#0  doadump () at pcpu.h:195
#1  0x0000000000000004 in ?? ()
#2  0xffffffff8050df19 in boot (howto=260)
    at /usr/src/sys/kern/kern_shutdown.c:418
#3  0xffffffff8050e322 in panic (fmt=0x104 <Address 0x104 out of bounds>)
    at /usr/src/sys/kern/kern_shutdown.c:574
#4  0xffffffff807d2193 in trap_fatal (frame=0xffffff0006abb000, eva=Variable "eva" is not available.
)
    at /usr/src/sys/amd64/amd64/trap.c:757
#5  0xffffffff807d2ce5 in trap (frame=0xffffffff51e2e360)
    at /usr/src/sys/amd64/amd64/trap.c:558
#6  0xffffffff807b700e in calltrap ()
    at /usr/src/sys/amd64/amd64/exception.S:209
#7  0xffffffff805bbc66 in rt_maskedcopy (src=0xffffffff51e2e6c8, 
    dst=0xffffff00525ebd80, netmask=0xef3fdf377db53afa)
    at /usr/src/sys/net/route.c:1362
#8  0xffffffff805bc4e5 in rtrequest1_fib (req=11, info=0xffffffff51e2e4c0, 
    ret_nrt=0xffffffff51e2e5e8, fibnum=0) at /usr/src/sys/net/route.c:1036
#9  0xffffffff805bd09d in rtrequest_fib (req=11, dst=0xffffffff51e2e6c8, 
    gateway=0x0, netmask=0x0, flags=0, ret_nrt=0xffffffff51e2e5e8, fibnum=0)
    at /usr/src/sys/net/route.c:738
#10 0xffffffff805bd531 in rtalloc1_fib (dst=0xffffffff51e2e6c8, report=1, 
    ignflags=18446744073709551615, fibnum=0) at /usr/src/sys/net/route.c:315
#11 0xffffffff805be749 in rtalloc_ign_fib (ro=0xffffffff51e2e6c0, ignore=0, 
    fibnum=0) at /usr/src/sys/net/route.c:252
#12 0xffffffff805f4cad in ip_output (m=0xffffff0006b04b00, opt=0x0, 
    ro=0xffffffff51e2e6c0, flags=0, imo=0x0, inp=0xffffff0006c41120)
    at /usr/src/sys/netinet/ip_output.c:230
#13 0xffffffff806582fa in tcp_output (tp=0xffffff0006c65b60)
    at /usr/src/sys/netinet/tcp_output.c:1128
#14 0xffffffff80663774 in tcp_usr_send (so=0xffffff0006aa85a0, flags=0, 
    m=0xffffff00526f3c00, nam=Variable "nam" is not available.
) at tcp_offload.h:269
#15 0xffffffff8056addb in sosend_generic (so=0xffffff0006aa85a0, addr=0x0, 
    uio=0x0, top=0xffffff00526f3c00, control=0x0, flags=Variable "flags" is not available.
)
    at /usr/src/sys/kern/uipc_socket.c:1246
#16 0xffffffff8069f73f in nfs_send (so=0xffffff0006aa85a0, nam=Variable "nam" is not available.
)
    at /usr/src/sys/nfsclient/nfs_socket.c:664
#17 0xffffffff806a2ab9 in nfs_request (vp=0xffffff0052bd9bd0, mrest=Variable "mrest" is not available.
)
    at /usr/src/sys/nfsclient/nfs_socket.c:1217
#18 0xffffffff806aadfa in nfs_readrpc (vp=0xffffff0052bd9bd0, 
    uiop=0xffffffff51e2eb30, cred=0xffffff0052899d00)
    at /usr/src/sys/nfsclient/nfs_vnops.c:1119
#19 0xffffffff8069a1c9 in nfs_doio (vp=0xffffff0052bd9bd0, 
    bp=0xffffffff26332020, cr=0xffffff0052899d00, td=Variable "td" is not available.
)
    at /usr/src/sys/nfsclient/nfs_bio.c:1571
#20 0xffffffff806a5e48 in nfssvc_iod (instance=Variable "instance" is not available.
)
    at /usr/src/sys/nfsclient/nfs_nfsiod.c:280
#21 0xffffffff804ea913 in fork_exit (callout=0xffffffff806a5c00 <nfssvc_iod>, 
    arg=0xffffffff80b4c880, frame=0xffffffff51e2ec80)
    at /usr/src/sys/kern/kern_fork.c:810
#22 0xffffffff807b73ce in fork_trampoline ()
    at /usr/src/sys/amd64/amd64/exception.S:455
#23 0x0000000000000000 in ?? ()
#24 0x0000000000000000 in ?? ()
#25 0x0000000000000001 in ?? ()
#26 0x0000000000000000 in ?? ()
#27 0x0000000000000000 in ?? ()
#28 0x0000000000000000 in ?? ()
[...]

>  If so, running 'l
> *0xffffffff80186249' (from the 'instruction pointer' line in the
> fault message) would be helpful.

This seems to point to crap... cam subsystem.
0xffffffff80186249 is in cam_periph_alloc (/usr/src/sys/cam/cam_periph.c:153)


I'll try to give you the lines from the panic above... This seems to make more sense.

(kgdb) l *0xffffffff805bbc66
0xffffffff805bbc66 is in rt_maskedcopy (/usr/src/sys/net/route.c:1366).
1361	rt_maskedcopy(struct sockaddr *src, struct sockaddr *dst, struct sockaddr *netmask)
1362	{
1363		register u_char *cp1 = (u_char *)src;
1364		register u_char *cp2 = (u_char *)dst;
1365		register u_char *cp3 = (u_char *)netmask;
1366		u_char *cplim = cp2 + *cp3;
1367		u_char *cplim2 = cp2 + *cp1;
1368	
1369		*cp2++ = *cp1++; *cp2++ = *cp1++; /* copies sa_len & sa_family */
1370		cp3 += 2;


I don't know what I can do to help you more. Message me, if you need more details.


I've disabled promiscuous mode now (disabled ipcad). First I/O tests showed
no panics. But the server has run for 4 days without problems last time, so I'm
going to let it run a bit longer.

--
Martin


More information about the freebsd-stable mailing list