panic: non-sleepable lock 6-STABLE

[Volker]

Hi!

This evening I've had a mysterious thing: Two reboots caused by a
panic in a row. The machine itself is running fine for months
(always on 6-STABLE). The last kernel / world build has been a week ago.

As I'm having KDB_UNATTENDED in my kernel, I didn't took notice of
that crash. Trying to inspect the crashdump does not give anything
useful:

> kgdb /usr/obj/usr/src/sys/BELLONA/kernel.debug /var/crash/vmcore.1
> [snip]
> This GDB was configured as "i386-marcel-freebsd".
> Cannot access memory at address 0xc08295f4
> (kgdb)

>From the messagelog I've been able to fish the following:

Sleeping thread (tid 100147, pid 1750) owns a non-sleepable lock
sched_switch(c3adbc00,0,1) at sched_switch+0x15b
mi_switch(1,0,c3adbc00,daab9a20,c05a657c,...) at mi_switch+0x1be
sleepq_switch(c36be500) at sleepq_switch+0x84
sleepq_wait(c36be500,0,c3adbc00,1,c36be500,...) at sleepq_wait+0x5c
msleep(c36be500,0,4c,c0766afa,0) at msleep+0x26f
usbd_transfer(c36be500,daab9aa0,c052b619,c36be500,c0598573,...) at
usbd_transfer
+0x145
usbd_sync_transfer(c36be500) at usbd_sync_transfer+0x11
usbd_do_request_flags_pipe(c3440780,c3440800,daab9afc,daab9afb,0,...)
at usbd_do
_request_flags_pipe+0x69
usbd_do_request_flags(c3440780,daab9afc,daab9afb,0,0,...) at
usbd_do_request_fla
gs+0x25
usbd_do_request(c3440780,daab9afc,daab9afb,ab9b14,f0c0,...) at
usbd_do_request+0
x1a
aue_csr_read_1(c3470600,0,0,0,80206932,...) at aue_csr_read_1+0x50
aue_setmulti(c3470600,c3adc100,c3b69a40,c346e800,daab9b68,...) at
aue_setmulti+0
x4d
aue_ioctl(c346e800,80206932,0) at aue_ioctl+0x106
if_delmulti(c346e800,c35ca1a0) at if_delmulti+0x203
in_delmulti_locked(c3adc0a0) at in_delmulti_locked+0x7b
in_delmulti(c3adc0a0) at in_delmulti+0x7b
ip_freemoptions(c3a37900,c36ebec4,0,c3b392c8,daab9bec,...) at
ip_freemoptions+0x
21
in_pcbdetach(c36ebec4) at in_pcbdetach+0x159
udp_detach(c3b392c8) at udp_detach+0xba
soclose(c3b392c8) at soclose+0xa1
soo_close(c3ae7798,c3adbc00) at soo_close+0x63
fdrop_locked(c3ae7798,c3adbc00,c3aeee00,daab9cb4,c056157b,...) at
fdrop_locked+0
xd8
fdrop(c3ae7798,c3adbc00,c058dde0,c09be140,c1de0668,...) at fdrop+0x40
closef(c3ae7798,c3adbc00,0,c3adbc00,6,...) at closef+0x43b
close(c3adbc00,daab9d04) at close+0x209
syscall(3b,3b,3b,2,807e6c0,...) at syscall+0x2cd
Xint0x80_syscall() at Xint0x80_syscall+0x1f
--- syscall (6, FreeBSD ELF32, close), eip = 0x281670ef, esp =
0xbfbfdcd0, ebp =
 0xbfbfdcdc ---
panic: sleeping thread
cpuid = 0
KDB: stack backtrace:
kdb_backtrace(100,c3adba80,c3adba80,c3adbc00,c3adbc00,...) at
kdb_backtrace+0x29
panic(c077339a,c3adbc00,ffffffff,c077335e,18733,...) at panic+0x113
propagate_priority(c3adba80,c35f81c0,c0805320,c080a8cc,c3adbc00,...)
at propagat
e_priority+0x58
turnstile_wait(c080a8cc,c3adbc00) at turnstile_wait+0x2e3
_mtx_lock_sleep(c080a8cc,c3adba80,0,0,0) at _mtx_lock_sleep+0x10c
udp_bind(c3bc9164,c35cab50,c3adba80,daab6cc0,c05c8177,...) at
udp_bind+0x34
sobind(c3bc9164,c35cab50,c3adba80,c3bef678,daab6d04,...) at sobind+0x16
kern_bind(c3adba80,5,c35cab50,c35cab50,c3adac90,...) at kern_bind+0x77
bind(c3adba80,daab6d04) at bind+0x2f
syscall(3b,3b,3b,80501a8,805f041,...) at syscall+0x2cd
Xint0x80_syscall() at Xint0x80_syscall+0x1f
--- syscall (104, FreeBSD ELF32, bind), eip = 0x2811877b, esp =
0xbfbfe76c, ebp
= 0xbfbfe7d8 ---
GEOM_MIRROR: Device gm0s2: rebuilding provider ad4s2 stopped.
Uptime: 8m50s
Dumping 511 MB (2 chunks)
  chunk 0: 1MB (159 pages) ... ok
  chunk 1: 511MB (130800 pages) 495 479 463 447 431 415 399 383 367
351 335 319
303 287 271 255 239 223 207 191 175 159 143 127 111 95 79 63 47 31
15 ... ok

I guess pid 1750 has been mDNSResponder
(/usr/ports/net/mDNSResponder - I found something in the logs which
indicates this might have been pid 1750 some time before the crash
occurs). I've installed this port just today (8 hours before the crash).

My main question now: Why didn't I get anything useful from the
savecore dumps?

I'll now prepare for a fresh kernel build, install and try to rerun
mdsnd to get a proper dump.

Greetings,

Volker

PS: I didn't include dmesg + config as I hate those lengthy postings
to the list. If a kernel hacker want's to take a closer look or
needs more infos, I'll put it up on the webserver for download if
it's of interest in this case.