debugging 6.1 crash
Stephen Clark
Stephen.Clark at seclark.us
Mon Feb 4 16:40:01 UTC 2008
Stephen Clark wrote:
> Hello List,
>
> I am trying to debug a 6.1 panic. When I run kgdb kernel.debug
> /var/crash/vmcore.7 all I get is:
>
> kgdb: kvm_read: invalid address (0x24)
> kgdb: kvm_read: invalid address (0x24)
> kgdb: kvm_read: invalid address (0x24)
> kgdb: kvm_read: invalid address (0x24)
> kgdb: kvm_read: invalid address (0x24)
> kgdb: kvm_read: invalid address (0x24)
> kgdb: kvm_read: invalid address (0x24)
> ...
>
> the info file shows:
> Dump header from device /dev/ad0s1b
> Architecture: i386
> Architecture Version: 2
> Dump Length: 116981760B (111 MB)
> Blocksize: 512
> Dumptime: Mon Feb 4 04:13:09 2008
> Hostname: G301482.netws.com
> Magic: FreeBSD Kernel Dump
> Version String: FreeBSD 6.1-STABLE #25: Wed Nov 14 10:30:01 EST 2007
> root at J301002.nwv01.com:/mnt/src/sys/i386/compile/WOLFPAC6SMP
> Panic String: page fault
> Dump Parity: 1156397610
> Bounds: 7
> Dump Status: good
>
>
> Does my kernel.debug have to match exactly the crash file kernel. I
> have made the following change
> to my kernel that the kernel.debug is based on.
> --- route.h.orig Tue Apr 4 22:07:23 2006
> +++ route.h Mon Dec 17 13:11:44 2007
> @@ -289,6 +289,7 @@
> #define RT_LOCK_INIT(_rt) \
> mtx_init(&(_rt)->rt_mtx, "rtentry", NULL, MTX_DEF | MTX_DUPOK)
> #define RT_LOCK(_rt) mtx_lock(&(_rt)->rt_mtx)
> +#define RT_TRYLOCK(_rt) mtx_trylock(&(_rt)->rt_mtx)
> #define RT_UNLOCK(_rt) mtx_unlock(&(_rt)->rt_mtx)
> #define RT_LOCK_DESTROY(_rt) mtx_destroy(&(_rt)->rt_mtx)
> #define RT_LOCK_ASSERT(_rt) mtx_assert(&(_rt)->rt_mtx,
> MA_OWNED)
> --- route.c.orig Tue Oct 30 19:07:54 2007
> +++ route.c Mon Dec 17 15:13:20 2007
> @@ -996,6 +996,7 @@
> struct radix_node_head *rnh = rt_tables[dst->sa_family];
> int dlen = SA_SIZE(dst), glen = SA_SIZE(gate);
>
> +again:
> RT_LOCK_ASSERT(rt);
>
> /*
> @@ -1029,7 +1030,15 @@
> RT_REMREF(rt);
> return (EADDRINUSE); /* failure */
> }
> - RT_LOCK(rt);
> + /*
> + * Try to reacquire the lock on rt, and if it fails,
> + * clean state and restart from scratch.
> + */
> + if (!RT_TRYLOCK(rt)) {
> + RTFREE_LOCKED(gwrt);
> + RT_LOCK(rt);
> + goto again;
> + }
> /*
> * If there is already a gwroute, then drop it. If we
> * are asked to replace route with itself, then do
>
> Thanks,
> Steve
>
Well I recompiled the kernel without the above changes and I am now kgdb
comes up. It looks like
the panic instruction pointer is in a loadable kernel module. Is there
some way to have kgdb look at the kernel
module?
Below is what i get now:
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i386-marcel-freebsd".
Unread portion of the kernel message buffer:
Fatal trap 12: page fault while in kernel mode
cpuid = 0; apic id = 00
fault virtual address = 0x4
fault code = supervisor read, page not present
instruction pointer = 0x20:0xc0b4536c
stack pointer = 0x28:0xc7516a30
frame pointer = 0x28:0xc7516a48
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, def32 1, gran 1
processor eflags = interrupt enabled, resume, IOPL = 0
current process = 13 (swi1: net)
trap number = 12
panic: page fault
cpuid = 0
Uptime: 8h31m18s
Dumping 111 MB (2 chunks)
chunk 0: 1MB (159 pages) ... ok
chunk 1: 111MB (28400 pages) 95 79 63 47 31 15
#0 doadump () at pcpu.h:165
165 __asm __volatile("movl %%fs:0,%0" : "=r" (td));
(kgdb) bt
#0 doadump () at pcpu.h:165
#1 0xc06492b2 in boot (howto=260) at ../../../kern/kern_shutdown.c:409
#2 0xc06495d9 in panic (fmt=0xc0910386 "%s") at
../../../kern/kern_shutdown.c:565
#3 0xc082d99c in trap_fatal (frame=0xc75169f0, eva=4) at
../../../i386/i386/trap.c:837
#4 0xc082d6db in trap_pfault (frame=0xc75169f0, usermode=0, eva=4)
at ../../../i386/i386/trap.c:745
#5 0xc082d335 in trap (frame=
{tf_fs = 8, tf_es = 40, tf_ds = 40, tf_edi = -950965440, tf_esi =
-1026657792, tf_ebp = -950965688, tf_isp = -950965732, tf_ebx =
-1045086208, tf_edx = -1047438316, tf_ecx = 0, tf_eax = 0, tf_trapno =
12, tf_err = 0, tf_eip = -1061923988, tf_cs = 32, tf_eflags = 590406,
tf_esp = -929974260, tf_ss = 0}) at ../../../i386/i386/trap.c:435
#6 0xc08198fa in calltrap () at ../../../i386/i386/exception.s:139
#7 0xc0b4536c in ?? ()
Cannot access memory at address 0xc891b80c
(kgdb) list *0xc0b4536c
No source file for address 0xc0b4536c.
(kgdb)
looking at the loadable kernel modules on the system I get:
sudo kldstat
Id Refs Address Size Name
1 13 0xc0400000 72862c kernel
2 1 0xc0b29000 2340 accf_http.ko
3 1 0xc0b2c000 3b180 ipf.ko
4 1 0xc0b68000 5c2f8 acpi.ko
5 1 0xc1dcc000 3000 ng_iface.ko
6 1 0xc1dcf000 6000 ng_ppp.ko
7 1 0xc1dd6000 4000 ng_bpf.ko
8 1 0xc1ddd000 4000 ng_vjc.ko
Which makes think the panic instruction pointer is in ipf.ko?
This is ipf 4.1.26 compiled out of the kernel tree.
Steve
--
"They that give up essential liberty to obtain temporary safety,
deserve neither liberty nor safety." (Ben Franklin)
"The course of history shows that as a government grows, liberty
decreases." (Thomas Jefferson)
More information about the freebsd-stable
mailing list