Server hanged on VFS lock problem

[Andrea Venturoli]

Hello.
Yesterday a 6.2/amd64 SMP server of mine entered DDB after a problem 
with locks. Since I was not there, I instructed one user to type "panic" 
and I got a dump.
Here's what I get:

> xxxxx# kgdb kernel.debug /var/crash/vmcore.0
> [GDB will not be able to debug user-mode threads: /usr/lib/libthread_db.so: Undefined symbol "ps_pglobal_lookup"]
> GNU gdb 6.1.1 [FreeBSD]
> Copyright 2004 Free Software Foundation, Inc.
> GDB is free software, covered by the GNU General Public License, and you are
> welcome to change it and/or distribute copies of it under certain conditions.
> Type "show copying" to see the conditions.
> There is absolutely no warranty for GDB.  Type "show warranty" for details.
> This GDB was configured as "amd64-marcel-freebsd".
> 
> Unread portion of the kernel message buffer:
> KDB: stack backtrace:
> vfs_badlock() at vfs_badlock+0x95
> assert_vop_elocked() at assert_vop_elocked+0x7d
> VOP_WRITE_APV() at VOP_WRITE_APV+0xee
> vn_write() at vn_write+0x1ec
> dofilewrite() at dofilewrite+0x87
> kern_writev() at kern_writev+0x51
> write() at write+0x4a
> syscall() at syscall+0x4d1
> Xfast_syscall() at Xfast_syscall+0xa8
> --- syscall (4, FreeBSD ELF64, write), rip = 0x8017ecbac, rsp = 0x7fffffffb568, rbp = 0x8054079c0 ---
> VOP_WRITE: 0xffffff00097f1ca8 is not exclusive locked but should be
> KDB: enter: lock violation
> Locked vnodes
> 
> 0xffffff00097f1ca8: tag ufs, type VREG
>     usecount 5, writecount 2, refcount 560 mountedhere 0
>     flags ()
>     v_object 0xffffff0002f65000 ref 3 pages 2216
>      lock type ufs: SHARED (count 1)#0 0xffffffff80238786 at lockmgr+0x5f6
> #1 0xffffffff8033d558 at ffs_lock+0x58
> #2 0xffffffff803b4df1 at VOP_LOCK_APV+0x81
> #3 0xffffffff802b80cb at vn_lock+0x6b
> #4 0xffffffff802b92c6 at vn_write+0x156
> #5 0xffffffff80271b37 at dofilewrite+0x87
> #6 0xffffffff80271e01 at kern_writev+0x51
> #7 0xffffffff80271efa at write+0x4a
> #8 0xffffffff803854a1 at syscall+0x4d1
> #9 0xffffffff80370b78 at Xfast_syscall+0xa8
> 
>         ino 1078094, on dev mirror/gm0s1e
> panic: from debugger
> cpuid = 0
> Uptime: 13d23h8m15s
> Dumping 1023 MB (2 chunks)
>   chunk 0: 1MB (151 pages) ... ok
>   chunk 1: 1023MB (261744 pages) 1007 991 975 959 943 927 911 895 879 863 847 831 815 799 783 767 751 735 719 703 687 671 655 639 623 607 591 575 559 543 527 511 495 479 463 447 431 415 399 383 367 351 335 319 303 287 271 255 239 223 207 191 175 159 143 127 111 95 79 63 47 31 15
> 
> #0  doadump () at pcpu.h:172
> 172             __asm __volatile("movq %%gs:0,%0" : "=r" (td));
> (kgdb)  


Any hint?
Is there a way I can get these dumps automatically, without entering 
DDB, since this is an unattended server?
I have this options in my kernel configuration:

options         KDB
options         DDB
options         KDB_UNATTENDED
options         INVARIANTS
options         INVARIANT_SUPPORT
options         WITNESS
options         DEBUG_LOCKS
options         DEBUG_VFS_LOCKS
options         DIAGNOSTIC

  bye & Thanks
	av.