6.2-RC2 panic - NFS client

Tue Jan 9 02:45:56 UTC 2007

Replying to myself...  This time triggered on a big portupgrade over NFS. 
I tried the "2 things hitting the same file" that triggered it the first 
time and it didn't take.  But a few minutes later while doing a giant 
portupgrade using packages from the NFS server I got another panic.

Let me know if there's anything else I can do.

Damn dwarves.

[root at spamd2 /usr/src]# kgdb -c /var/crash/vmcore.1 
/boot/kernel/kernel.debug
kgdb: kvm_nlist(_stopped_cpus):
kgdb: kvm_nlist(_stoppcbs):
[GDB will not be able to debug user-mode threads: 
/usr/lib/libthread_db.so: Undefined symbol "ps_pglobal_lookup"]
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you 
are welcome to change it and/or distribute copies of it under certain 
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for 
details.
This GDB was configured as "i386-marcel-freebsd".

Unread portion of the kernel message buffer:

Fatal trap 12: page fault while in kernel mode
fault virtual address   = 0x34
fault code              = supervisor read, page not present
instruction pointer     = 0x20:0xc05eab55
stack pointer           = 0x28:0xe7aba7d0
frame pointer           = 0x28:0xe7aba7f0
code segment            = base 0x0, limit 0xfffff, type 0x1b
                         = DPL 0, pres 1, def32 1, gran 1
processor eflags        = interrupt enabled, resume, IOPL = 0
current process         = 3397 (bsdtar)
trap number             = 12
panic: page fault
Uptime: 1h7m41s
Dumping 1015 MB (2 chunks)
   chunk 0: 1MB (160 pages) ... ok
   chunk 1: 1015MB (259840 pages) 1000 984 968 952 936 920 904 888 872 856 
840 824 808 792 776 760 744 728 712 696 680 664 648 632 616 600 584 568 
552 536 520 504 488 472 456 440 424 408 392 376 360 344 328 312 296 280 
264 248 232 216 200 184 168 152 136 120 104 88 72 56 40 24 8

#0  doadump () at pcpu.h:165
165     pcpu.h: No such file or directory.
         in pcpu.h
(kgdb) where
#0  doadump () at pcpu.h:165
#1  0xc0595634 in boot (howto=260) at 
/usr/src/sys/kern/kern_shutdown.c:409
#2  0xc0595966 in panic (fmt=0xc078b204 "%s")
     at /usr/src/sys/kern/kern_shutdown.c:565
#3  0xc075ecfc in trap_fatal (frame=0xe7aba790, eva=0)
     at /usr/src/sys/i386/i386/trap.c:837
#4  0xc075ea02 in trap_pfault (frame=0xe7aba790, usermode=0, eva=52)
     at /usr/src/sys/i386/i386/trap.c:745
#5  0xc075e5cd in trap (frame=
       {tf_fs = 8, tf_es = 40, tf_ds = 40, tf_edi = -657625352, tf_esi = 
-657625352, tf_ebp = -408180752, tf_isp = -408180804, tf_ebx = -657625352, 
tf_edx = 4, tf_ecx = -988422784, tf_eax = 0, tf_trapno = 12, tf_err = 0, 
tf_eip = -1067537579, tf_cs = 32, tf_eflags = 66178, tf_esp = 17104896, 
tf_ss = 48})
     at /usr/src/sys/i386/i386/trap.c:435
#6  0xc074ab6a in calltrap () at /usr/src/sys/i386/i386/exception.s:139
#7  0xc05eab55 in vfs_vmio_release (bp=0xd8cd6ef8) at atomic.h:146
#8  0xc05eb54e in getnewbuf (slpflag=0, slptimeo=0, size=2048, 
maxsize=16384)
     at /usr/src/sys/kern/vfs_bio.c:1779
#9  0xc05ecf31 in getblk (vp=0xc53bf990, blkno=0, size=2048, slpflag=0,
     slptimeo=0, flags=0) at /usr/src/sys/kern/vfs_bio.c:2497
#10 0xc06c5a4e in ffs_balloc_ufs2 (vp=0xc53bf990, startoffset=Unhandled 
dwarf expression opcode 0x93
)
     at /usr/src/sys/ufs/ffs/ffs_balloc.c:676
#11 0xc06eeab1 in ufs_mkdir (ap=0xe7ababac)
     at /usr/src/sys/ufs/ufs/ufs_vnops.c:1587
#12 0xc0773d33 in VOP_MKDIR_APV (vop=0x0, a=0xe7ababac) at vnode_if.c:1251
#13 0xc060a8f3 in kern_mkdir (td=0xc515dd80,
     path=0x805a440 <Address 0x805a440 out of bounds>, 
segflg=UIO_USERSPACE, mode=448) at vnode_if.h:653
#14 0xc060a549 in mkdir (td=0x0, uap=0x4)
     at /usr/src/sys/kern/vfs_syscalls.c:3388
#15 0xc075f0d2 in syscall (frame=
       {tf_fs = 59, tf_es = 59, tf_ds = 59, tf_edi = 134587456, tf_esi = 
134587473, tf_ebp = -1077942216, tf_isp = -408179356, tf_ebx = 671761748, 
tf_edx = 0, tf_ecx = 134574127, tf_eax = 136, tf_trapno = 0, tf_err = 2, 
tf_eip = 672701275, tf_cs = 51, tf_eflags = 582, tf_esp = -1077942372, 
tf_ss = 59})
     at /usr/src/sys/i386/i386/trap.c:983
#16 0xc074abbf in Xint0x80_syscall () at 
/usr/src/sys/i386/i386/exception.s:200
#17 0x00000033 in ?? ()
Previous frame inner to this frame (corrupt stack?)
(kgdb)

On Mon, 8 Jan 2007, Charles Sprickman wrote:

> Hi all,
>
> Just killed what had been a fairly stable 6.2-RC2 box with NFS.
>
> I have four boxes that run spamd (spamassassin spamd, not pf spamd). Since 
> they are all now in sync and running 6.2-RC2, I set one up as an NFS server 
> to make updates across all four boxes easier.  I got a little bit ahead of 
> myself and was running a 'pkgdb -F' on both the server and one of the 
> clients.  The process on the server just errored out and exited.  The client 
> spit this out and then paniced:
>
> database file error
> [Updating the portsdb <format:bdb_btree> in /usr/ports ... - 14186 port 
> entries found ......../usr/ports/INDEX-6:830:read: 0x8752584, 1024: Stale NFS 
> file handle -- Stale NFS file handle
> /usr/ports/INDEX-6:831:read: 0x8752584, 1024: Stale NFS file handle -- Stale 
> NFS file handle
> /usr/ports/INDEX-6:832:read: 0x8752584, 1024: Stale NFS file handle -- Stale 
> NFS file handle
> /usr/ports/INDEX-6:833:read: 0x8752584, 1024: Stale NFS file handle -- Stale 
> NFS file handle
> /usr/ports/INDEX-6:834:read: 0x8752584, 1024: Stale NFS file handle -- Stale 
> NFS file handle
> /usr/ports/INDEX-6:835:read: 0x8752584, 1024: Stale NFS file handle -- Stale 
> NFS file handle
> /usr/ports/INDEX-6:836:read: 0x8752584, 1024: Stale NFS file handle -- Stale 
> NFS file handle
> Connection to spamd2 closed.
>
> I have the kernel dump, but have two problems getting more info:
>
> -I had just clobbered /usr/obj to make some room, not realizing that's where 
> the kernel.debug ends up (why is it in there?).
>
> -I can't find the current "how to provide a useful panic report" document in 
> the handbook that I use for reference.
>
> I can probably make this thing crash again if there's any interest.
>
> Thanks,
>
> Charles
>
> _______________________________________________
> freebsd-stable at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscribe at freebsd.org"
>