kern/68442: panic - acquiring duplicate lock of same type: "sleepq chain"

[Daniel Lang]

Hi again,

Ok, here are some LOR's that occurred today on the machine
before it paniced and wedged. The LOR's are not yet listed
on the LOR page, so I cc: Bjoern.

The panic and subsequent system wedge did not happen
immediately after the LORs, the system continued to
run for a while, but only for short while after the second LOR.

Important INFO: I cvsuped and built new world and kernel
(in single user mode, the system appears to be able to do
some work in single user) of _today_. I hoped the problem might
go away. I also removed KVA_PAGES=512 from kernel config,
so default KVA_PAGES apply. It apparently did not help.

LOR1:
[..]
login: lock order reversal
 1st 0xc070a0c0 sched lock (sched lock) @ /usr/src/sys/kern/kern_proc.c:672
 2nd 0xc0745d40 sio (sio) @ /usr/src/sys/dev/sio/sio.c:3205
Stack backtrace:
backtrace(ffffffff,c0712948,c0712b00,c06e25c4,c07358dc) at 0xc051e066 = backtra2
witness_checkorder(c0745d40,9,c06b1e34,c85,3f8) at 0xc05393c4 = witness_checkor4
_mtx_lock_spin_flags(c0745d40,0,c06b1e34,c85,c0712498) at 0xc0516e9e = _mtx_loce
siocnputc(c06f9c40,63) at 0xc064375f = siocnputc+0x6b
cnputc(63) at 0xc05483ed = cnputc+0x4d
putchar(63,e5bd87e0) at 0xc053387a = putchar+0x92
kvprintf(c069d236,c05337e8,e5bd87e0,a,e5bd8800) at 0xc0533a87 = kvprintf+0x77
printf(c069d236,32cf1,0,32ce6,0,dd8,c9115828) at 0xc0533763 = printf+0x43
calcru(c91156e0,e5bd8ad0,e5bd8ad8,0,e5bd8a10) at 0xc051cb9e = calcru+0x1f2
fill_kinfo_thread(c436e930,e5bd88fc,e5bd8b98,c05194b6,c91156e0) at 0xc0518f2a =6
fill_kinfo_proc(c91156e0,e5bd88fc,dd8,288,0) at 0xc0518c01 = fill_kinfo_proc+0x1
sysctl_out_proc(c91156e0,e5bd8c08,4,0,4) at 0xc05194b6 = sysctl_out_proc+0x32
sysctl_kern_proc(c06e2c20,e5bd8c90,0,e5bd8c08,c06e2c20) at 0xc05199d8 = sysctl_4
sysctl_root(0,e5bd8c84,3,e5bd8c08,ca65fe70) at 0xc052519f = sysctl_root+0x10f
userland_sysctl(ca65fe70,e5bd8c84,3,0,bfbfe47c) at 0xc052535c = userland_sysctlc
__sysctl(ca65fe70,e5bd8d14,6,3,296) at 0xc052521d = __sysctl+0x71
syscall(2f,2f,2f,3,0) at 0xc0664713 = syscall+0x217
Xint0x80_syscall() at 0xc06539ff = Xint0x80_syscall+0x1f
--- syscall (202), eip = 0x280dd05b, esp = 0xbfbfe40c, ebp = 0xbfbfe448 ---
[..]

LOR2:
[..]
acquiring duplicate lock of same type: "sleepq chain"
 1st Giant @ /usr/src/sys/kern/uipc_syscalls.c:1735
 2nd sleepq chain @ /usr/src/sys/kern/subr_sleepqueue.c:223
Stack backtrace:
backtrace(c053a384,c0712970,c0712970,c06e25c4,c07359bc) at 0xc051e066 = backtra2
witness_checkorder(c070ea1c,9,c069f25e,df,398) at 0xc05393c4 = witness_checkord4
_mtx_lock_spin_flags(c070ea1c,0,c069f25e,df,c3f96150) at 0xc0516e9e = _mtx_locke
sleepq_lookup(c9f29724,c8cc0abc,0,c069f25e,16b) at 0xc053675e = sleepq_lookup+0e
sleepq_catch_signals(c9f29724,0,100,0,c06a1bac) at 0xc05369f1 = sleepq_catch_sid
msleep(c9f29724,c9f296f4,158,c06a1a05,0) at 0xc052375b = msleep+0x233
sbwait(c9f296e0,0,51d,0,0) at 0xc054f90f = sbwait+0x33
do_sendfile(c3f96150,e5400d14,0,e5400d40,c0664713) at 0xc055385d = do_sendfile+5
sendfile(c3f96150,e5400d14,8,8,202) at 0xc0552bfc = sendfile+0x10
syscall(806002f,2819002f,bfbf002f,8,0) at 0xc0664713 = syscall+0x217
Xint0x80_syscall() at 0xc06539ff = Xint0x80_syscall+0x1f
--- syscall (393), eip = 0x2812123b, esp = 0xbfbfde4c, ebp = 0xbfbfdea8 ---
[..]

And here the panic message:
[..]
Fatal trap 12: page fault while in kernel mode
cpuid = 1; apic id = 06
fault virtual address   = 0x34
fault code              = supervisor read, page not present
instruction pointer     = 0x8:0xc053932b
stack pointer           = 0x10:0xe53d8ab0
frame pointer           = 0x10:0xe53d8ad4
code segment            = base 0x0, limit 0xfffff, type 0x1b
                        = DPL 0, pres 1, def32 1, gran 1
processor eflags        = resume, IOPL = 0
current process         = 2550 (cvsupd)
[..]

No ddb prompt, no crash-dump, no reboot. I need to go and
reset the thing (now for the dozenth time :-/).

Thanks and best regards,
 Daniel
-- 
IRCnet: Mr-Spock     - Cool people don't move, they just hang around. -  
Daniel Lang * dl at leo.org * ++49 89 289 18532  * http://www.leo.org/~dl/