kernel page fault panic in yesterday's current ( nfs_nget -> vfs_hash_insert -> bcmp )

[Don Lewis]

I got this panic when running cvs to cvs update my /usr/ports tree in
preparation for updating to today's kernel, world, and ports.

Mlock order reversal
 1st 0xc0957d20 vfs hash (vfs hash) @ /usr/src/sys/kern/vfs_hash.c:115
 2nd 0xc29fa170 user map (user map) @ /usr/src/sys/vm/vm_map.c:2998
KDB: stack backtrace:
kdb_backtrace(0,ffffffff,c09196b8,c091a680,c08a646c) at kdb_backtrace+0x29
witness_checkorder(c29fa170,9,c085ca65,bb6) at witness_checkorder+0x54c
_sx_xlock(c29fa170,c085ca65,bb6) at _sx_xlock+0x50
_vm_map_lock_read(c29fa12c,c085ca65,bb6,1000046,c502ae4c) at _vm_map_lock_read+0
x37
vm_map_lookup(eb5fc858,0,1,eb5fc85c,eb5fc84c) at vm_map_lookup+0x28
vm_fault(c29fa12c,0,1,0,c29da000) at vm_fault+0x66
trap_pfault(eb5fc920,0,0) at trap_pfault+0xf2
trap(18,10,10,c570686c,0) at trap+0x335
calltrap() at calltrap+0x5
--- trap 0xc, eip = 0xc07d6d16, esp = 0xeb5fc960, ebp = 0xeb5fc978 ---
bcmp(c30b5b18,eb5fc9cc) at bcmp+0x16
vfs_hash_insert(c30b5b18,cd563ea,2,c29da000,eb5fc9c8) at vfs_hash_insert+0x7a
nfs_nget(c2483400,c570686c,1c,eb5fca14,c29da000) at nfs_nget+0x179
nfs_lookup(eb5fcb24,c31218e0,c2625c21,eb5fcb40,c066d554) at nfs_lookup+0x96d
VOP_LOOKUP_APV(c08b4780,eb5fcb24) at VOP_LOOKUP_APV+0x87
lookup(eb5fcc10,0,0,c29da000,0) at lookup+0x354
namei(eb5fcc10,c08455bc,69f,c0910dc0,0) at namei+0x346
kern_stat(c29da000,80f6000,0,eb5fcc84) at kern_stat+0x35
stat(c29da000,eb5fcd14,2,29af,282) at stat+0x1b
syscall(2f,2f,2f,80f4026,80f4021) at syscall+0x213
Xint0x80_syscall() at Xint0x80_syscall+0x1f
--- syscall (188, FreeBSD ELF32, stat), eip = 0x482cb3bb, esp = 0xbfbfe4ec, ebp 
= 0xbfbfe568 ---


Fatal trap 12: page fault while in kernel mode
cpuid = 0; apic id = 00
fault virtual address   = 0x0
fault code              = supervisor read, page not present
instruction pointer     = 0x8:0xc07d6d16
stack pointer           = 0x10:0xeb5fc960
frame pointer           = 0x10:0xeb5fc978
code segment            = base 0x0, limit 0xfffff, type 0x1b
                        = DPL 0, pres 1, def32 1, gran 1
processor eflags        = interrupt enabled, resume, IOPL = 0
current process         = 42904 (cvs)
[thread pid 42904 tid 100130 ]
Stopped at      bcmp+0x16:      repe cmpsl      (%esi),%es:(%edi)



kgdb seems to be having trouble unwinding the stack:

#9  0xc07c62aa in calltrap () at /usr/src/sys/i386/i386/exception.s:139
#10 0x00000018 in ?? ()
#11 0x00000010 in ?? ()
---Type <return> to continue, or q <return> to quit---
#12 0x00000010 in ?? ()
#13 0xc570686c in ?? ()
#14 0x00000000 in ?? ()
#15 0xeb5fc978 in ?? ()
#16 0xeb5fc94c in ?? ()
#17 0xc588fd50 in ?? ()
#18 0x0000001c in ?? ()
#19 0x00000007 in ?? ()
#20 0x00000000 in ?? ()
#21 0x0000000c in ?? ()
#22 0x00000000 in ?? ()
#23 0xc07d6d16 in bcmp () at /usr/src/sys/i386/i386/support.s:1408
#24 0x0cd563ea in ?? ()
#25 0xc30b5b18 in ?? ()
#26 0xc06ef904 in nfs_vncmpf (vp=0x0, arg=0x0)
    at /usr/src/sys/nfsclient/nfs_node.c:92
#27 0xc066c656 in vfs_hash_insert (vp=0x8, hash=3229445398, flags=2, 
    td=0xc29da000, vpp=0xeb5fc9c8, fn=0xc06ef8e8 <nfs_vncmpf>, arg=0xeb5fc9cc)
    at /usr/src/sys/kern/vfs_hash.c:122
#28 0xc06efa81 in nfs_nget (mntp=0xc2483400, fhp=0xc570686c, fhsize=28, 
    npp=0xeb5fca14) at /usr/src/sys/nfsclient/nfs_node.c:161
#29 0xc06f69f5 in nfs_lookup (ap=0x0) at /usr/src/sys/nfsclient/nfs_vnops.c:920
#30 0xc07eb34f in VOP_LOOKUP_APV (vop=0xc08b4780, a=0xeb5fcb24)


The kernel is fairly close to GENERIC, with no special compile flags.

Copyright (c) 1992-2005 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
FreeBSD 6.0-CURRENT #302: Wed Mar 16 15:06:44 PST 2005
    dl at scratch.catspoiler.org:/usr/obj/usr/src/sys/GENERICSMB
WARNING: WITNESS option enabled, expect reduced performance.
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: Unknown CPU Type (2010.29-MHz 686-class CPU)
  Origin = "AuthenticAMD"  Id = 0x681  Stepping = 1
  Features=0x383fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CM
OV,PAT,PSE36,MMX,FXSR,SSE>
  AMD Features=0xc0400000<AMIE,DSP,3DNow!>
real memory  = 1073676288 (1023 MB)
avail memory = 1041948672 (993 MB)
ACPI APIC Table: <GBT    AWRDACPI>
ioapic0 <Version 1.1> irqs 0-23 on motherboard
npx0: [FAST]
npx0: <math processor> on motherboard
npx0: INT 16 interface
acpi0: <GBT AWRDACPI> on motherboard
acpi0: Power Button (fixed)
Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x4008-0x400b on acpi0
cpu0: <ACPI CPU> on acpi0


I'm going to go ahead with the kernel and world update and see if the
problem is reproduceable afterwards.