UFS related panic (daily <-> find)

Wed Oct 16 10:49:17 UTC 2013

I've:
Read whole disk with 'dd'
Checked FS integrity from single user mode.
Did a RAM test for ~5 hours (3 complete tests passed in a loop) via 
memtest86+.

All passed without errors ...
Now I'm posting this:

FreeBSD 9.2-RELEASE #0 r256015
i386
======================================================
Fatal trap 12: page fault while in kernel mode
fault virtual address   = 0x25
fault code              = supervisor read, page not present
instruction pointer     = 0x20:0xc083bd42
stack pointer           = 0x28:0xe7bfa7ac
frame pointer           = 0x28:0xe7bfa7b0
code segment            = base 0x0, limit 0xfffff, type 0x1b
                        = DPL 0, pres 1, def32 1, gran 1
processor eflags        = interrupt enabled, resume, IOPL = 0
current process         = 39060 (find)
trap number             = 12
panic: page fault
Uptime: 1d6h37m19s
panic: bufwrite: buffer is not busy???
Uptime: 1d6h37m19s
Sleeping thread (tid 100122, pid 39060) owns a non-sleepable lock
panic: sleeping thread
Uptime: 1d6h37m19s
Physical memory: 1014 MB
Dumping 162 MB: 147 131 115 99 83 67 51 35 19 3

No symbol "stopped_cpus" in current context.
No symbol "stoppcbs" in current context.
Reading symbols from /boot/kernel/nfscl.ko...Reading symbols from 
/boot/kernel/nfscl.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/nfscl.ko
Reading symbols from /boot/kernel/nfslock.ko...Reading symbols from 
/boot/kernel/nfslock.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/nfslock.ko
Reading symbols from /boot/kernel/nfssvc.ko...Reading symbols from 
/boot/kernel/nfssvc.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/nfssvc.ko
Reading symbols from /boot/kernel/krpc.ko...Reading symbols from 
/boot/kernel/krpc.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/krpc.ko
Reading symbols from /boot/kernel/nfscommon.ko...Reading symbols from 
/boot/kernel/nfscommon.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/nfscommon.ko
Reading symbols from /boot/kernel/nfsd.ko...Reading symbols from 
/boot/kernel/nfsd.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/nfsd.ko
Reading symbols from /boot/kernel/nfslockd.ko...Reading symbols from 
/boot/kernel/nfslockd.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/nfslockd.ko
Reading symbols from /usr/local/modules/fuse.ko...done.
Loaded symbols for /usr/local/modules/fuse.ko
Reading symbols from /usr/local/lib/oss/modules/osscore.ko...done.
Loaded symbols for /usr/local/lib/oss/modules/osscore.ko
Reading symbols from /usr/local/lib/oss/modules/oss_audigyls.ko...done.
Loaded symbols for /usr/local/lib/oss/modules/oss_audigyls.ko
Reading symbols from /boot/kernel/cpuctl.ko...Reading symbols from 
/boot/kernel/cpuctl.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/cpuctl.ko
#0  doadump (textdump=1) at pcpu.h:249
249     pcpu.h: No such file or directory.
        in pcpu.h
(kgdb) #0  doadump (textdump=1) at pcpu.h:249
#1  0xc065edaf in kern_reboot (howto=260)
    at /usr/src/sys/kern/kern_shutdown.c:449
#2  0xc065efe1 in panic (fmt=<value optimized out>)
    at /usr/src/sys/kern/kern_shutdown.c:637
#3  0xc08bb30a in trap_fatal (frame=0xe7bfa76c, eva=37)
    at /usr/src/sys/i386/i386/trap.c:1044
#4  0xc08bb3f1 in trap_pfault (frame=0xe7bfa76c, usermode=0, eva=37)
    at /usr/src/sys/i386/i386/trap.c:896
#5  0xc08bc0a0 in trap (frame=0xe7bfa76c) at 
/usr/src/sys/i386/i386/trap.c:555
#6  0xc08a94ec in calltrap () at /usr/src/sys/i386/i386/exception.s:170
#7  0xc083bd42 in inodedep_find (inodedephd=<value optimized out>,
    fs=0xc4758800, inum=1130832, inodedeppp=0xe7bfa7fc)
    at /usr/src/sys/ufs/ffs/ffs_softdep.c:2131
#8  0xc0840520 in inodedep_lookup (mp=0xc479d7ec, inum=1130832, flags=0,
    inodedeppp=0xe7bfa7fc) at /usr/src/sys/ufs/ffs/ffs_softdep.c:2163
#9  0xc0844e5a in softdep_load_inodeblock (ip=0xc81c1910)
    at /usr/src/sys/ufs/ffs/ffs_softdep.c:11735
#10 0xc08532b2 in ffs_vgetf (mp=0xc479d7ec, ino=1130832, flags=2097152,
    vpp=0xe7bfa8f0, ffs_flags=<value optimized out>)
    at /usr/src/sys/ufs/ffs/ffs_vfsops.c:1773
#11 0xc08533cd in ffs_vget (mp=0xc479d7ec, ino=1130832, flags=2097152,
    vpp=0xe7bfa8f0) at /usr/src/sys/ufs/ffs/ffs_vfsops.c:1656
#12 0xc08602c0 in ufs_lookup_ino (vdp=0xc85aa9fc, vpp=0xe7bfaae8,
    cnp=0xe7bfaafc, dd_ino=0x0) at /usr/src/sys/ufs/ufs/ufs_lookup.c:749
#13 0xc08602fb in ufs_lookup (ap=0xe7bfa964)
    at /usr/src/sys/ufs/ufs/ufs_lookup.c:214
#14 0xc08d4736 in VOP_CACHEDLOOKUP_APV (vop=0xc0998b40, a=0xe7bfa964)
    at vnode_if.c:193
#15 0xc06d675f in vfs_cache_lookup (ap=0xe7bfa9ec) at vnode_if.h:80
#16 0xc08d47dc in VOP_LOOKUP_APV (vop=0xc0998b40, a=0xe7bfa9ec)
    at vnode_if.c:126
#17 0xc06ddf93 in lookup (ndp=0xe7bfaabc) at vnode_if.h:54
#18 0xc06defcf in namei (ndp=0xe7bfaabc) at 
/usr/src/sys/kern/vfs_lookup.c:294
#19 0xc06f1089 in kern_statat_vnhook (td=0xc4b1e2f0, flag=512, fd=-100,
    path=0x284512b8 <Address 0x284512b8 out of bounds>,
    pathseg=UIO_USERSPACE, sbp=0xe7bfabe8, hook=0)
    at /usr/src/sys/kern/vfs_syscalls.c:2432
#20 0xc06f11d2 in kern_statat (td=0xc4b1e2f0, flag=512, fd=-100,
    path=0x284512b8 <Address 0x284512b8 out of bounds>,
    pathseg=UIO_USERSPACE, sbp=0xe7bfabe8)
    at /usr/src/sys/kern/vfs_syscalls.c:2413
#21 0xc06f120a in kern_lstat (td=0xc4b1e2f0,
    path=0x284512b8 <Address 0x284512b8 out of bounds>,
    pathseg=UIO_USERSPACE, sbp=0xe7bfabe8)
    at /usr/src/sys/kern/vfs_syscalls.c:2486
#22 0xc06f129e in sys_lstat (td=0xc4b1e2f0, uap=0xe7bfaccc)
    at /usr/src/sys/kern/vfs_syscalls.c:2476
#23 0xc08bb8bb in syscall (frame=0xe7bfad08) at subr_syscall.c:135
#24 0xc08a9551 in Xint0x80_syscall ()
    at /usr/src/sys/i386/i386/exception.s:270
#25 0x00000033 in ?? ()
Previous frame inner to this frame (corrupt stack?)
(kgdb)
======================================================

Domagoj