Fatal trap 12: page fault while in kernel mode

[Yuri Khotyaintsev]

I have the following panic occurring several times a week. The machine is an 
NFS server, and it usually panics early in the morning, when first people try 
to access it. After reboot it may work OK for 1-2 days, and then panics 
again. I have tried changing memory and replacing disk which was exported via 
NFS, but nothing helped :(

Any suggestion on how to fix this panic will be very much appreciated ! 

/Yuri

[root at XXX][/var/crash]# uname -a
FreeBSD XXX.irfu.se 6.0-STABLE FreeBSD 6.0-STABLE #0: Tue Nov 29 13:31:15 CET 
2005     root at XXX.irfu.se:/usr/obj/usr/src/sys/HEM  i386
[root at XXX][/var/crash]# kgdb /usr/obj/usr/src/sys/HEM/kernel.debug vmcore.7
[GDB will not be able to debug user-mode threads: /usr/lib/libthread_db.so: 
Undefined symbol "ps_pglobal_lookup"]
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-marcel-freebsd".

Unread portion of the kernel message buffer:
kernel trap 12 with interrupts disabled


Fatal trap 12: page fault while in kernel mode
fault virtual address	= 0x74
fault code		= supervisor read, page not present
instruction pointer	= 0x20:0xc053a426
stack pointer	        = 0x28:0xd56c0b88
frame pointer	        = 0x28:0xd56c0b8c
code segment		= base 0x0, limit 0xfffff, type 0x1b
			= DPL 0, pres 1, def32 1, gran 1
processor eflags	= resume, IOPL = 0
current process		= 77 (vnlru)
trap number		= 12
panic: page fault
Uptime: 2d12h22m11s
Dumping 511 MB (2 chunks)
  chunk 0: 1MB (160 pages) ... ok
  chunk 1: 511MB (130800 pages) 495 479 463 447 431 415 399 383 367 351 335 
319 303 287 271 255 239 223 207 191 175 159 143 127 111 95 79 63 47 31 15

#0  doadump () at pcpu.h:165
165	pcpu.h: No such file or directory.
	in pcpu.h
(kgdb) where
#0  doadump () at pcpu.h:165
#1  0xc051577a in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:399
#2  0xc0515a84 in panic (fmt=0xc06ce475 "%s") 
at /usr/src/sys/kern/kern_shutdown.c:555
#3  0xc06b4815 in trap_fatal (frame=0xd56c0b48, eva=0)
    at /usr/src/sys/i386/i386/trap.c:836
#4  0xc06b3f2d in trap (frame=
      {tf_fs = 1133445128, tf_es = 40, tf_ds = 40, tf_edi = -1017997312, 
tf_esi = -1020120704, tf_ebp = -714339444, tf_isp = -714339468, tf_ebx = 
-1012942272, tf_edx = -1020120704, tf_ecx = 0, tf_eax = 0, tf_trapno = 12, 
tf_err = 0, tf_eip = -1068260314, tf_cs = 32, tf_eflags = 589831, tf_esp = 
-1020120704, tf_ss = -714339408})
    at /usr/src/sys/i386/i386/trap.c:269
#5  0xc06a24fa in calltrap () at /usr/src/sys/i386/i386/exception.s:139
#6  0xc053a426 in turnstile_setowner (ts=0xc39fba40, owner=0x0)
    at /usr/src/sys/kern/subr_turnstile.c:417
#7  0xc053a752 in turnstile_wait (lock=0xc461fe00, owner=0x0)
    at /usr/src/sys/kern/subr_turnstile.c:576
#8  0xc050b511 in _mtx_lock_sleep (m=0xc461fe00, tid=3274846592, opts=0, 
file=0x0, line=0)
    at /usr/src/sys/kern/kern_mutex.c:555
#9  0xc064becd in ufsdirhash_free (ip=0xc4a33840)
    at /usr/src/sys/ufs/ufs/ufs_dirhash.c:289
#10 0xc064de66 in ufs_reclaim (ap=0x0) at /usr/src/sys/ufs/ufs/ufs_inode.c:175
#11 0xc06bef38 in VOP_RECLAIM_APV (vop=0x0, a=0xc3323180) at vnode_if.c:1589
#12 0xc057adfe in vgonel (vp=0xc3cf3aa0) at vnode_if.h:818
#13 0xc0577530 in vtryrecycle (vp=0xc3cf3aa0) 
at /usr/src/sys/kern/vfs_subr.c:840
#14 0xc0576ec6 in vnlru_free (count=1376) at /usr/src/sys/kern/vfs_subr.c:668
#15 0xc0577019 in vnlru_proc () at /usr/src/sys/kern/vfs_subr.c:703
#16 0xc04fc310 in fork_exit (callout=0xc0576f24 <vnlru_proc>, arg=0x0, 
frame=0x0)
    at /usr/src/sys/kern/kern_fork.c:789
#17 0xc06a255c in fork_trampoline () at /usr/src/sys/i386/i386/exception.s:208
(kgdb) quit

-- 
Dr. Yuri Khotyaintsev
Institutet för rymdfysik (IRF), Uppsala