threads/132215: Crash after running ppp

[John Baldwin]

The following reply was made to PR kern/132215; it has been noted by GNATS.

From: John Baldwin <jhb at freebsd.org>
To: freebsd-threads at freebsd.org
Cc: Stephane Legrand <stephleg at free.fr>, freebsd-gnats-submit at freebsd.org,
        rwatson at freebsd.org
Subject: Re: threads/132215: Crash after running ppp
Date: Mon, 2 Mar 2009 11:11:32 -0500

 On Sunday 01 March 2009 4:56:21 am Stephane Legrand wrote:
 > 
 > >Number:         132215
 > >Category:       threads
 > >Synopsis:       Crash after running ppp
 > >Confidential:   no
 > >Severity:       serious
 > >Priority:       low
 > >Responsible:    freebsd-threads
 > >State:          open
 > >Quarter:        
 > >Keywords:       
 > >Date-Required:
 > >Class:          sw-bug
 > >Submitter-Id:   current-users
 > >Arrival-Date:   Sun Mar 01 10:00:09 UTC 2009
 > >Closed-Date:
 > >Last-Modified:
 > >Originator:     Stephane Legrand
 > >Release:        7.1-STABLE
 > >Organization:
 > >Environment:
 > 7.1-STABLE i386
 > >Description:
 > Hi,
 > 
 > After trying to update my system to the 7.1-STABLE of 28th february 2009, 
 when i run ppp, it crashes the system. Here is the backtrace : 
 > 
 > sequoia# kgdb kernel.debug /var/crash/vmcore.0
 > GNU gdb 6.1.1 [FreeBSD]
 > Copyright 2004 Free Software Foundation, Inc.
 > GDB is free software, covered by the GNU General Public License, and you are
 > welcome to change it and/or distribute copies of it under certain 
 conditions.
 > Type "show copying" to see the conditions.
 > There is absolutely no warranty for GDB.  Type "show warranty" for details.
 > This GDB was configured as "i386-marcel-freebsd"...
 > 
 > Unread portion of the kernel message buffer:
 > Sleeping thread (tid 100053, pid 516) owns a non-sleepable lock
 > panic: sleeping thread
 > cpuid = 0
 > Uptime: 37s
 > Physical memory: 1002 MB
 > Dumping 158 MB: 143 127 111 95 79 63 47 31 15
 > 
 > Reading symbols from /boot/kernel/linprocfs.ko...Reading symbols 
 from /boot/kernel/linprocfs.ko.symbols...done.
 > done.
 > Loaded symbols for /boot/kernel/linprocfs.ko
 > Reading symbols from /boot/kernel/linux.ko...Reading symbols 
 from /boot/kernel/linux.ko.symbols...done.
 > done.
 > Loaded symbols for /boot/kernel/linux.ko
 > Reading symbols from /boot/kernel/linsysfs.ko...Reading symbols 
 from /boot/kernel/linsysfs.ko.symbols...done.
 > done.
 > Loaded symbols for /boot/kernel/linsysfs.ko
 > Reading symbols from /boot/kernel/snd_hda.ko...Reading symbols 
 from /boot/kernel/snd_hda.ko.symbols...done.
 > done.
 > Loaded symbols for /boot/kernel/snd_hda.ko
 > Reading symbols from /boot/kernel/sound.ko...Reading symbols 
 from /boot/kernel/sound.ko.symbols...done.
 > done.
 > Loaded symbols for /boot/kernel/sound.ko
 > Reading symbols from /boot/kernel/acpi.ko...Reading symbols 
 from /boot/kernel/acpi.ko.symbols...done.
 > done.
 > Loaded symbols for /boot/kernel/acpi.ko
 > Reading symbols from /boot/kernel/netgraph.ko...Reading symbols 
 from /boot/kernel/netgraph.ko.symbols...done.
 > done.
 > Loaded symbols for /boot/kernel/netgraph.ko
 > Reading symbols from /boot/kernel/ng_ether.ko...Reading symbols 
 from /boot/kernel/ng_ether.ko.symbols...done.
 > done.
 > Loaded symbols for /boot/kernel/ng_ether.ko
 > Reading symbols from /boot/kernel/ng_pppoe.ko...Reading symbols 
 from /boot/kernel/ng_pppoe.ko.symbols...done.
 > done.
 > Loaded symbols for /boot/kernel/ng_pppoe.ko
 > Reading symbols from /boot/kernel/ng_socket.ko...Reading symbols 
 from /boot/kernel/ng_socket.ko.symbols...done.
 > done.
 > Loaded symbols for /boot/kernel/ng_socket.ko
 > Reading symbols from /boot/kernel/ipfw.ko...Reading symbols 
 from /boot/kernel/ipfw.ko.symbols...done.
 > done.
 > Loaded symbols for /boot/kernel/ipfw.ko
 > Reading symbols from /usr/local/modules/fuse.ko...done.
 > Loaded symbols for /usr/local/modules/fuse.ko
 > Reading symbols from /boot/kernel/accf_http.ko...Reading symbols 
 from /boot/kernel/accf_http.ko.symbols...done.
 > done.
 > Loaded symbols for /boot/kernel/accf_http.ko
 > Reading symbols from /boot/kernel/i915.ko...done.
 > Loaded symbols for /boot/kernel/i915.ko
 > Reading symbols from /boot/kernel/drm.ko...done.
 > Loaded symbols for /boot/kernel/drm.ko
 > #0  doadump () at pcpu.h:196
 > 196		__asm __volatile("movl %%fs:0,%0" : "=r" (td));
 > (kgdb) bt
 > #0  doadump () at pcpu.h:196
 > #1  0xc075c6df in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:418
 > #2  0xc075c9a4 in panic (fmt=Variable "fmt" is not available.
 > ) at /usr/src/sys/kern/kern_shutdown.c:574
 > #3  0xc078eeec in propagate_priority (td=0xc444a690) 
 at /usr/src/sys/kern/subr_turnstile.c:222
 > #4  0xc078fd11 in turnstile_wait (ts=0xc4260e60, owner=0xc444a690, 
 queue=Variable "queue" is not available.
 > ) at /usr/src/sys/kern/subr_turnstile.c:740
 > #5  0xc074f6b3 in _mtx_lock_sleep (m=0xc4267e7c, tid=3297722368, opts=0, 
 file=0x0, line=0)
 >     at /usr/src/sys/kern/kern_mutex.c:420
 
 Can you do something like this in kgdb:
 
 frame 5
 set $td = (struct thread *)(m->mtx_lock & ~3)
 p $td->td_tid
 
 (This should print out a thread ID)
 
 thread $td->td_tid
 where
 
 This should give you the stack trace of the misbehaving thread which slept 
 while holding a route lock.
 
 Also, this is a kernel bug, not a thread issue.
 
 -- 
 John Baldwin