setrunqueue: corrupt kq_runq, td=0xc2745c80 (Sept 17ish kernel)

[Robert Watson]

This panic occurred while I was doing a make depend over NFS.  mysqld was
presumably largely idle, but perhaps woke up to do some processing on a
timer.  This kernel is a GENERIC kernel from around September 17; I was in
the process of building an updated kernel when it panicked, so it could be
that this is fixed already? 

db> show msgbuf
msgbufp = 0xc101bfe4
magic = 63062, size = 32740, r= 9434, w = 9540, ptr = 0xc1014000, cksum=
743102
setrunqueue(): corrupt kq_runq, td= 0xc2745c80
panic: deadlock in setrunqueue
cpuid = 0
KDB: enter: panic
Copyright (c) 1992-2004 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
...
db> show pcpu
cpuid        = 0
curthread    = 0xc67944b0: pid 565 "mysqld"
curpcb       = 0xef63ada0
fpcurthread  = none
idlethread   = 0xc2260960: pid 14 "idle: cpu0"
APIC ID      = 0
currentldt   = 0x30
spin locks held:
db> trace
kdb_enter(c07f0956) at kdb_enter+0x2b
panic(c07f1f58,c07f1f30,c2745c80,0,c2745c80) at panic+0x127
setrunqueue(c2745c80,0) at setrunqueue+0x11c
turnstile_unpend(c29494c0,c296a06c,39f,ef63ac68,c05f79e8) at
turnstile_unpend+0x274
_mtx_unlock_sleep(c296a06c,0,c07ee778,39f) at _mtx_unlock_sleep+0x6c
_mtx_unlock_flags(c296a06c,0,c07ee778,39f) at _mtx_unlock_flags+0x98
thread_link_mboxes(c22d7a10,c2b2e030,0,0,c22d7a10) at
thread_link_mboxes+0xcf
thread_userret(c67944b0,ef63ad48) at thread_userret+0x374
userret(c67944b0,ef63ad48,b7,1,b7) at userret+0x57
syscall(830002f,2f,a89002f,8304a00,0) at syscall+0x2d9
Xint0x80_syscall() at Xint0x80_syscall+0x1f
--- syscall (0, FreeBSD ELF32, nosys), eip = 0x2833ad6f, esp = 0xa893f88,
ebp = 0xa893fc4 ---

Robert N M Watson             FreeBSD Core Team, TrustedBSD Projects
robert at fledge.watson.org      Principal Research Scientist, McAfee Research