threads/132215: Crash after running ppp

[John Baldwin]

On Sunday 01 March 2009 4:56:21 am Stephane Legrand wrote:
> 
> >Number:         132215
> >Category:       threads
> >Synopsis:       Crash after running ppp
> >Confidential:   no
> >Severity:       serious
> >Priority:       low
> >Responsible:    freebsd-threads
> >State:          open
> >Quarter:        
> >Keywords:       
> >Date-Required:
> >Class:          sw-bug
> >Submitter-Id:   current-users
> >Arrival-Date:   Sun Mar 01 10:00:09 UTC 2009
> >Closed-Date:
> >Last-Modified:
> >Originator:     Stephane Legrand
> >Release:        7.1-STABLE
> >Organization:
> >Environment:
> 7.1-STABLE i386
> >Description:
> Hi,
> 
> After trying to update my system to the 7.1-STABLE of 28th february 2009, 
when i run ppp, it crashes the system. Here is the backtrace : 
> 
> sequoia# kgdb kernel.debug /var/crash/vmcore.0
> GNU gdb 6.1.1 [FreeBSD]
> Copyright 2004 Free Software Foundation, Inc.
> GDB is free software, covered by the GNU General Public License, and you are
> welcome to change it and/or distribute copies of it under certain 
conditions.
> Type "show copying" to see the conditions.
> There is absolutely no warranty for GDB.  Type "show warranty" for details.
> This GDB was configured as "i386-marcel-freebsd"...
> 
> Unread portion of the kernel message buffer:
> Sleeping thread (tid 100053, pid 516) owns a non-sleepable lock
> panic: sleeping thread
> cpuid = 0
> Uptime: 37s
> Physical memory: 1002 MB
> Dumping 158 MB: 143 127 111 95 79 63 47 31 15
> 
> Reading symbols from /boot/kernel/linprocfs.ko...Reading symbols 
from /boot/kernel/linprocfs.ko.symbols...done.
> done.
> Loaded symbols for /boot/kernel/linprocfs.ko
> Reading symbols from /boot/kernel/linux.ko...Reading symbols 
from /boot/kernel/linux.ko.symbols...done.
> done.
> Loaded symbols for /boot/kernel/linux.ko
> Reading symbols from /boot/kernel/linsysfs.ko...Reading symbols 
from /boot/kernel/linsysfs.ko.symbols...done.
> done.
> Loaded symbols for /boot/kernel/linsysfs.ko
> Reading symbols from /boot/kernel/snd_hda.ko...Reading symbols 
from /boot/kernel/snd_hda.ko.symbols...done.
> done.
> Loaded symbols for /boot/kernel/snd_hda.ko
> Reading symbols from /boot/kernel/sound.ko...Reading symbols 
from /boot/kernel/sound.ko.symbols...done.
> done.
> Loaded symbols for /boot/kernel/sound.ko
> Reading symbols from /boot/kernel/acpi.ko...Reading symbols 
from /boot/kernel/acpi.ko.symbols...done.
> done.
> Loaded symbols for /boot/kernel/acpi.ko
> Reading symbols from /boot/kernel/netgraph.ko...Reading symbols 
from /boot/kernel/netgraph.ko.symbols...done.
> done.
> Loaded symbols for /boot/kernel/netgraph.ko
> Reading symbols from /boot/kernel/ng_ether.ko...Reading symbols 
from /boot/kernel/ng_ether.ko.symbols...done.
> done.
> Loaded symbols for /boot/kernel/ng_ether.ko
> Reading symbols from /boot/kernel/ng_pppoe.ko...Reading symbols 
from /boot/kernel/ng_pppoe.ko.symbols...done.
> done.
> Loaded symbols for /boot/kernel/ng_pppoe.ko
> Reading symbols from /boot/kernel/ng_socket.ko...Reading symbols 
from /boot/kernel/ng_socket.ko.symbols...done.
> done.
> Loaded symbols for /boot/kernel/ng_socket.ko
> Reading symbols from /boot/kernel/ipfw.ko...Reading symbols 
from /boot/kernel/ipfw.ko.symbols...done.
> done.
> Loaded symbols for /boot/kernel/ipfw.ko
> Reading symbols from /usr/local/modules/fuse.ko...done.
> Loaded symbols for /usr/local/modules/fuse.ko
> Reading symbols from /boot/kernel/accf_http.ko...Reading symbols 
from /boot/kernel/accf_http.ko.symbols...done.
> done.
> Loaded symbols for /boot/kernel/accf_http.ko
> Reading symbols from /boot/kernel/i915.ko...done.
> Loaded symbols for /boot/kernel/i915.ko
> Reading symbols from /boot/kernel/drm.ko...done.
> Loaded symbols for /boot/kernel/drm.ko
> #0  doadump () at pcpu.h:196
> 196		__asm __volatile("movl %%fs:0,%0" : "=r" (td));
> (kgdb) bt
> #0  doadump () at pcpu.h:196
> #1  0xc075c6df in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:418
> #2  0xc075c9a4 in panic (fmt=Variable "fmt" is not available.
> ) at /usr/src/sys/kern/kern_shutdown.c:574
> #3  0xc078eeec in propagate_priority (td=0xc444a690) 
at /usr/src/sys/kern/subr_turnstile.c:222
> #4  0xc078fd11 in turnstile_wait (ts=0xc4260e60, owner=0xc444a690, 
queue=Variable "queue" is not available.
> ) at /usr/src/sys/kern/subr_turnstile.c:740
> #5  0xc074f6b3 in _mtx_lock_sleep (m=0xc4267e7c, tid=3297722368, opts=0, 
file=0x0, line=0)
>     at /usr/src/sys/kern/kern_mutex.c:420

Can you do something like this in kgdb:

frame 5
set $td = (struct thread *)(m->mtx_lock & ~3)
p $td->td_tid

(This should print out a thread ID)

thread $td->td_tid
where

This should give you the stack trace of the misbehaving thread which slept 
while holding a route lock.

Also, this is a kernel bug, not a thread issue.

-- 
John Baldwin