kern/117182: 7.0-PRERELEASE with ULE: panic: sleeping thread

[Stefan Krüger]

>Number:         117182
>Category:       kern
>Synopsis:       7.0-PRERELEASE with ULE: panic: sleeping thread
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Sun Oct 14 14:10:01 UTC 2007
>Closed-Date:
>Last-Modified:
>Originator:     Stefan Krüger
>Release:        FreeBSD 7.0-PRERELEASE i386
>Organization:
>Environment:
FreeBSD localhost 7.0-PRERELEASE FreeBSD 7.0-PRERELEASE #1: Sun Oct 14 03:30:36 CEST 2007     skrueger at localhost:/usr/obj/usr/src/sys/SMP_ULE  i386
>Description:
$ kgdb /boot/kernel/kernel.symbols /var/crash/vmcore.2
[...]
Unread portion of the kernel message buffer:
Sleeping thread (tid 100047, pid 97714) owns a non-sleepable lock
panic: sleeping thread
cpuid = 0
Uptime: 3h42m19s
Physical memory: 1015 MB
Dumping 189 MB: 174 158 142 126 110 94 78 62 46 30 14

#0  doadump () at pcpu.h:195
195     pcpu.h: No such file or directory.
        in pcpu.h
(kgdb) bt
#0  doadump () at pcpu.h:195
#1  0xc05159c6 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:409
#2  0xc0515c9e in panic (fmt=Variable "fmt" is not available.
) at /usr/src/sys/kern/kern_shutdown.c:563
#3  0xc054817c in propagate_priority (td=0xc3c5caa0)
    at /usr/src/sys/kern/subr_turnstile.c:222
#4  0xc0549045 in turnstile_wait (ts=0xc3adfd20, owner=0xc3c5caa0, queue=Variable "queue" is not available.
)
    at /usr/src/sys/kern/subr_turnstile.c:739
#5  0xc05094da in _mtx_lock_sleep (m=0xc07f3e6c, tid=3283112480, opts=0, 
    file=0x0, line=0) at /usr/src/sys/kern/kern_mutex.c:394
#6  0xc062647e in tcp_input (m=0xc402c300, off0=20)
    at /usr/src/sys/netinet/tcp_input.c:400
#7  0xc05c80a0 in ip_input (m=0xc402c300)
    at /usr/src/sys/netinet/ip_input.c:665
#8  0xc05b3392 in netisr_dispatch (num=2, m=0xc402c300)
    at /usr/src/sys/net/netisr.c:185
#9  0xc05af8b1 in ether_demux (ifp=0xc3bcec00, m=0xc402c300)
    at /usr/src/sys/net/if_ethersubr.c:834
#10 0xc05afcdf in ether_input (ifp=0xc3bcec00, m=0xc402c300)
    at /usr/src/sys/net/if_ethersubr.c:692
#11 0xc047047a in fxp_intr (xsc=0xc3ba7000)
    at /usr/src/sys/dev/fxp/if_fxp.c:1706
#12 0xc04f88cb in ithread_loop (arg=0xc3c2c340)
    at /usr/src/sys/kern/kern_intr.c:1036
#13 0xc04f5554 in fork_exit (callout=0xc04f8700 <ithread_loop>, 
    arg=0xc3c2c340, frame=0xe2603d38) at /usr/src/sys/kern/kern_fork.c:796
#14 0xc0724740 in fork_trampoline () at /usr/src/sys/i386/i386/exception.s:205


(kgdb) info threads
[...]
    at /usr/src/sys/kern/sched_ule.c:1908
  90 Thread 100147 (PID=97714: squid)  sched_switch (td=0xc6fb8000, newtd=Variable "newtd" is not available.
)
[...]
(kgdb) thread 90
[Switching to thread 90 (Thread 100147)]#0  sched_switch (td=0xc6fb8000, newtd=Variable "newtd" is not available.

) at /usr/src/sys/kern/sched_ule.c:1908
1908                    cpuid = PCPU_GET(cpuid);
(kgdb) bt
#0  sched_switch (td=0xc6fb8000, newtd=Variable "newtd" is not available.
) at /usr/src/sys/kern/sched_ule.c:1908
#1  0xc051db46 in mi_switch (flags=Variable "flags" is not available.
) at /usr/src/sys/kern/kern_synch.c:442
#2  0xc0545478 in sleepq_switch (wchan=Variable "wchan" is not available.
)
    at /usr/src/sys/kern/subr_sleepqueue.c:459
#3  0xc054573a in sleepq_catch_signals (wchan=0xc6820280)
    at /usr/src/sys/kern/subr_sleepqueue.c:414
#4  0xc0545fdd in sleepq_wait_sig (wchan=0xc6820280)
    at /usr/src/sys/kern/subr_sleepqueue.c:556
#5  0xc051df87 in _sleep (ident=0xc6820280, lock=0xc07e7cc0, priority=256, 
    wmesg=0xc07748da "ucond", timo=0) at /usr/src/sys/kern/kern_synch.c:226
#6  0xc0529dca in __umtx_op_cv_wait (td=0xc6fb8000, uap=0xe669bcfc)
    at /usr/src/sys/kern/kern_umtx.c:467
#7  0xc0527ee7 in _umtx_op (td=0xc6fb8000, uap=0xe669bcfc)
    at /usr/src/sys/kern/kern_umtx.c:2490
#8  0xc073d3b5 in syscall (frame=0xe669bd38)
    at /usr/src/sys/i386/i386/trap.c:1008
#9  0xc0724730 in Xint0x80_syscall () at /usr/src/sys/i386/i386/exception.s:196
#10 0x00000033 in ?? ()
Previous frame inner to this frame (corrupt stack?)

>How-To-Repeat:
- install squid with aufs support
- setup squid using aufs
- start a webbrowser (firefox in my case) and open many sites at once (e.g. using Bookmarks -> * -> "Open All in Tabs")


- kernel + vmcore available on request
>Fix:


>Release-Note:
>Audit-Trail:
>Unformatted: