Page fault in kernel mode; unable to dump core; reproducable

Tai-hwa Liang avatar at mmlab.cse.yzu.edu.tw
Mon Jun 14 02:34:10 GMT 2004


On Sun, 13 Jun 2004, Peter Schuller wrote:
> Hello,
>
> i am experiencing a reproducable 'page fault while in kernel mode' with
> CURRENT (from a couple of weeks ago aswell as one cvsuped today). It happens
> extremely early during boot right after the root filesystem is mounted. This
> does not happen with 5.2.1-RELEASE.

That's what I saw on my T40. However, this only happens if I boot with a
GENERIC kernel instead of a custom one.

> I have tried setting 'dumpdev' appropriately (/dev/ad0s2b in my case), but no
> crash dumps are written (not automatically, and not in response to the
> 'panic' command in the kernel debugger). What should I do in order to provide
> the necessary information about this bug?

I'm not sure about that; however, I can get the backtrace reliably by
issuing a 'panic' command in DDB.

> The bug *may* be triggered by the fact that the root filesystem has been
> mounted read-write since the last crash that originally marked the file
> system dirty. I do this sometimes to get around the fact that the startup
> sequence won't defer the root fs for bgfsck until it's been mounted rw at
> least once since being marked dirty.

Perhaps this is also related to the thread "Thinkpad panic woes (was Re:
CDRW causes Thinkpad T41 to panic)" about two weeks ago?

http://lists.freebsd.org/pipermail/freebsd-current/2004-June/028173.html

>
> If I boot into single user mount it will survive mounting the root filesystem
> ro. It has also survived, at least once, remounting rw, but then crashed when
> I did a 'sync'. On another occasion, I let it boot to single user mode,
> mounting ro, after which it crashed when I did 'sysctl -a | grep dumpdev'.
>
> So. I have kept the filesystem dirty, interrupting the bg fsck_ufs when
> booting with 5.2.1 in order to maintain the state which triggers the bug. I
> had meant to post a full stack trace but since I am unable to obtain a dump,
> I will at least include what's on the screen even though it's probably
> useless:
>
> ==
> Memory modified after free 0xc1c08600(508) val=1000100 @ 0xc1c08600
>
> Fatal trap 12: page fault while in kernel mode
> fault virtual address = 0x1000120
> fault code               = supervisor read, page not present
> instruction pointer    = 0x8 :0xc063d63e
> stack pointer           = 0x10 :0xd5469984
> frame pointer          = 0x10 :0xd54699a0
> code segment         = base 0x0, limit 0xfffff, type 0x1b
>                              = DPL 0, pres 1, def32 1, gran 1
> processor eflags      = interrupt enabled, resume, IOPL = 0
> current process       = 64 (sh)
> kernel: type 12 trap, code=0
> Stopped at       mtrash_ctor+0x3a:     movl   0x20(%eax),%eax

It looks like the second panic message around line 138 triggered this panic
"earlier" -- the content of *ksp is trashed; therefore, the reference
through (*ksp) to ks_shortdesc caused this panic.

# cvsup'ed on Jun-11-2004 CST
# Memory modified after free 0xc1740a00(508) val=1000100 @ 0xc1740a00
# kernel configuration file: /sys/i386/conf/GENERIC
#
panic: from debugger
panic messages:
---
Fatal trap 12: page fault while in kernel mode
cpuid = 0; apic id = 00
fault virtual address	= 0x1000120
fault code		= supervisor read, page not present
instruction pointer	= 0x8:0xc072cf4e
stack pointer	        = 0x10:0xdd153984
frame pointer	        = 0x10:0xdd1539a0
code segment		= base 0x0, limit 0xfffff, type 0x1b
			= DPL 0, pres 1, def32 1, gran 1
processor eflags	= interrupt enabled, resume, IOPL = 0
current process		= 624 (csh)
kernel: type 12 trap, code=0


Fatal trap 12: page fault while in kernel mode
cpuid = 0; apic id = 00
fault virtual address	= 0x1000120
fault code		= supervisor read, page not present
instruction pointer	= 0x8:0xc072cf4e
stack pointer	        = 0x10:0xdd153984
frame pointer	        = 0x10:0xdd1539a0
code segment		= base 0x0, limit 0xfffff, type 0x1b
			= DPL 0, pres 1, def32 1, gran 1
processor eflags	= interrupt enabled, resume, IOPL = 0
current process		= 624 (csh)
kernel: type 12 trap, code=0
exclusive sleep mutex Giant r = 0 (0xc0888140) locked @ /home/avatar/ncvs/src/sys/vm/vm_map.c:1393
panic: from debugger
cpuid = 0;


Fatal trap 3: breakpoint instruction fault while in kernel mode
cpuid = 0; apic id = 00
instruction pointer	= 0x8:0xc074ea3e
stack pointer	        = 0x10:0xdd153768
frame pointer	        = 0x10:0xdd15376c
code segment		= base 0x0, limit 0xfffff, type 0x1b
			= DPL 0, pres 1, def32 1, gran 1
processor eflags	= IOPL = 0
current process		= 624 (csh)
panic: from debugger
cpuid = 0;
Uptime: 1m17s
Dumping 255 MB
 16 32 48 64 80 96 112 128 144 160 176 192 208 224 240
---
Reading symbols from /boot/kernel/linux.ko...done.
Loaded symbols for /boot/kernel/linux.ko
Reading symbols from /boot/kernel/if_em.ko...done.
Loaded symbols for /boot/kernel/if_em.ko
Reading symbols from /boot/kernel/if_wi.ko...done.
Loaded symbols for /boot/kernel/if_wi.ko
Reading symbols from /boot/kernel/snd_ich.ko...done.
Loaded symbols for /boot/kernel/snd_ich.ko
Reading symbols from /boot/kernel/snd_pcm.ko...done.
Loaded symbols for /boot/kernel/snd_pcm.ko
Reading symbols from /boot/kernel/ums.ko...done.
Loaded symbols for /boot/kernel/ums.ko
Reading symbols from /boot/kernel/umass.ko...done.
Loaded symbols for /boot/kernel/umass.ko
Reading symbols from /boot/kernel/if_ath.ko...done.
Loaded symbols for /boot/kernel/if_ath.ko
Reading symbols from /boot/kernel/ath_hal.ko...done.
Loaded symbols for /boot/kernel/ath_hal.ko
Reading symbols from /boot/kernel/smbfs.ko...done.
Loaded symbols for /boot/kernel/smbfs.ko
Reading symbols from /boot/kernel/libmchain.ko...done.
Loaded symbols for /boot/kernel/libmchain.ko
Reading symbols from /boot/kernel/libiconv.ko...done.
Loaded symbols for /boot/kernel/libiconv.ko
Reading symbols from /boot/kernel/radeon.ko...done.
Loaded symbols for /boot/kernel/radeon.ko
Reading symbols from /boot/kernel/acpi.ko...done.
Loaded symbols for /boot/kernel/acpi.ko
Reading symbols from /boot/kernel/msdosfs_iconv.ko...done.
Loaded symbols for /boot/kernel/msdosfs_iconv.ko
Reading symbols from /boot/kernel/linprocfs.ko...done.
Loaded symbols for /boot/kernel/linprocfs.ko
#0  doadump () at /home/avatar/ncvs/src/sys/kern/kern_shutdown.c:236
236		dumping++;
(kgdb) where
#0  doadump () at /home/avatar/ncvs/src/sys/kern/kern_shutdown.c:236
#1  0xc05eba4c in boot (howto=260)
    at /home/avatar/ncvs/src/sys/kern/kern_shutdown.c:370
#2  0xc05ebd77 in panic ()
    at /home/avatar/ncvs/src/sys/kern/kern_shutdown.c:548
#3  0xc045d985 in db_panic () at /home/avatar/ncvs/src/sys/ddb/db_command.c:453
#4  0xc045d91c in db_command (last_cmdp=0xc086b6e0, cmd_table=0xc07f6400,
    aux_cmd_tablep=0xc07ed508, aux_cmd_tablep_end=0xc07ed520)
    at /home/avatar/ncvs/src/sys/ddb/db_command.c:348
#5  0xc045d9f4 in db_command_loop ()
    at /home/avatar/ncvs/src/sys/ddb/db_command.c:475
#6  0xc0460179 in db_trap (type=12, code=0)
    at /home/avatar/ncvs/src/sys/ddb/db_trap.c:73
#7  0xc074e781 in kdb_trap (type=12, code=0, regs=0xdd153944)
    at /home/avatar/ncvs/src/sys/i386/i386/db_interface.c:159
#8  0xc0761563 in trap_fatal (frame=0xdd153944, eva=16777504)
    at /home/avatar/ncvs/src/sys/i386/i386/trap.c:810
#9  0xc07612a7 in trap_pfault (frame=0xdd153944, usermode=0, eva=16777504)
    at /home/avatar/ncvs/src/sys/i386/i386/trap.c:733
#10 0xc0760f09 in trap (frame=
      {tf_fs = 24, tf_es = 16, tf_ds = 16, tf_edi = 0, tf_esi = -1049359872, tf_ebp = -585811552, tf_isp = -585811600, tf_ebx = -1049359364, tf_edx = 0, tf_ecx = -1056882688, tf_eax = 16777472, tf_trapno = 12, tf_err = 0, tf_eip = -10662176
50, tf_cs = 8, tf_eflags = 66054, tf_esp = -1065474503, tf_ss = -1049359872})
    at /home/avatar/ncvs/src/sys/i386/i386/trap.c:420
#11 0xc072cf4e in mtrash_ctor (mem=0xc1740a00, size=0, arg=0x0)
    at /home/avatar/ncvs/src/sys/vm/uma_dbg.c:137
#12 0xc072b9cc in uma_zalloc_arg (zone=0xc10359a0, udata=0x0, flags=2)
    at /home/avatar/ncvs/src/sys/vm/uma_core.c:1642
#13 0xc05e29ca in malloc (size=3238288352, type=0xc0822a40, flags=2)
    at /home/avatar/ncvs/src/sys/vm/uma.h:270
#14 0xc05c9b09 in elf32_load_file (p=0xc19c4c08, file=0x0, addr=0xdd153ab0,
    entry=0x0, pagesize=4096)
    at /home/avatar/ncvs/src/sys/kern/imgact_elf.c:518
#15 0xc05ca307 in exec_elf32_imgact (imgp=0xdd153b94)
    at /home/avatar/ncvs/src/sys/kern/imgact_elf.c:827
#16 0xc05d582a in kern_execve (td=0xc19c6b00, fname=---Can't read userspace from dump, or kernel process---)
    at /home/avatar/ncvs/src/sys/kern/kern_exec.c:382
#17 0xc05d54cc in execve (td=0xc19c6b00, uap=0x0)
    at /home/avatar/ncvs/src/sys/kern/kern_exec.c:174
#18 0xc0761833 in syscall (frame=
      {tf_fs = 47, tf_es = 47, tf_ds = 47, tf_edi = 135295360, tf_esi = 135063552, tf_ebp = -1077958264, tf_isp = -585810572, tf_ebx = 672975340, tf_edx = 135063559, tf_ecx = 672975340, tf_eax = 59, tf_trapno = 22, tf_err = 2, tf_eip = 672504151, tf_cs = 31, tf_eflags = 514, tf_esp = -1077958292, tf_ss = 47})
    at /home/avatar/ncvs/src/sys/i386/i386/trap.c:1004
#19 0x28159957 in ?? ()
---Can't read userspace from dump, or kernel process---

(kgdb) f 11
#11 0xc072cf4e in mtrash_ctor (mem=0xc1740a00, size=0, arg=0x0)
    at /home/avatar/ncvs/src/sys/vm/uma_dbg.c:137
137				panic("Most recently used by %s\n", (*ksp == NULL)?
(kgdb) l
132
133		for (p = mem; cnt > 0; cnt--, p++)
134			if (*p != uma_junk) {
135				printf("Memory modified after free %p(%d) val=%x @ %p\n",
136				    mem, size, *p, p);
137				panic("Most recently used by %s\n", (*ksp == NULL)?
138				    "none" : (*ksp)->ks_shortdesc);
139			}
140	}
141
(kgdb) print ksp
$1 = (struct malloc_type **) 0xc1740bfc
(kgdb) print *ksp
$2 = (struct malloc_type *) 0x1000100
(kgdb) print *ksp->ks_shortdesc
---Can't read userspace from dump, or kernel process---


More information about the freebsd-current mailing list