Kernel panic of "spin lock held too long"

Fri Jun 25 13:56:22 UTC 2010

I met the kernel dump again.The message is below.

/*  -------------------------- dump message start  ----------------------------------------------*/
fangli# kgdb  /boot/kernel.old/kernel /var/crash/vmcore.7
GNU gdb 6.1.1 [FreeBSD]
Copyright  2004 Free Software Foundation, Inc.
GDB is free software, covered by  the GNU General Public License, and you are
welcome to change it  and/or distribute copies of it under certain conditions.
Type "show  copying" to see the conditions.
There is absolutely no warranty for  GDB.  Type "show warranty" for details.
This GDB was configured as  "i386-marcel-freebsd"...

Unread portion of the kernel message  buffer:
spin lock 0xc0d9e580 (sched lock 1) held by 0xc5546b40 (tid  100003) too long
panic: spin lock held too long
cpuid = 0
Uptime:  5h1m27s
Physical memory: 2034 MB
Dumping 136 MB: 121 105 89 73 57  41 25 9

Reading symbols from  /boot/kernel.old/./zaptel.ko...done.
Loaded symbols for  /boot/kernel.old/./zaptel.ko
Reading symbols from  /boot/kernel.old/./tej21.ko...done.
Loaded symbols for  /boot/kernel.old/./tej21.ko
#0  doadump () at pcpu.h:246
246      pcpu.h: No such file or directory.
        in pcpu.h
(kgdb) bt
#0   doadump () at pcpu.h:246
#1  0xc087cf67 in boot (howto=260) at  /usr/src/sys/kern/kern_shutdown.c:416
#2  0xc087d259 in panic  (fmt=Variable "fmt" is not available.
) at  /usr/src/sys/kern/kern_shutdown.c:579
#3  0xc086d50f in  _mtx_lock_spin_failed (m=0x0) at /usr/src/sys/kern/kern_mutex.c:490
#4   0xc086d595 in _mtx_lock_spin (m=0xc0d9e580, tid=3339617408, opts=0,  file=0x0, line=0)
    at /usr/src/sys/kern/kern_mutex.c:526
#5   0xc089edf8 in sched_add (td=0xc5546b40, flags=0) at  /usr/src/sys/kern/sched_ule.c:1109
#6  0xc08b6dc4 in turnstile_unpend  (ts=0xc5526d80, owner_type=0) at /usr/src/sys/kern/subr_turnstile.c:931
#7   0xc086d4b1 in _mtx_unlock_sleep (m=0xc5e6518c, opts=0, file=0xc5e4830a  "zaptel-base.c", line=3299)
    at /usr/src/sys/kern/kern_mutex.c:684
#8   0xc086d97f in _mtx_unlock_flags (m=0xc5e6518c, opts=0, file=0xc5e4830a  "zaptel-base.c", line=3299)
    at /usr/src/sys/kern/kern_mutex.c:227
#9   0xc5e44e7e in zt_ioctl () from /boot/kernel.old/./zaptel.ko
#10  0xc0804627 in devfs_ioctl_f (fp=0xc598d508, com=2147764778,  data=0xc5970270, cred=0xc5f08080, td=0xc70e8480)
    at  /usr/src/sys/fs/devfs/devfs_vnops.c:659
#11 0xc08b8e00 in kern_ioctl  (td=0xc70e8480, fd=92, com=2147764778, data=0xc5970270 "\001") at  file.h:262
#12 0xc08b8f74 in ioctl (td=0xc70e8480, uap=0xe7d2acf8) at  /usr/src/sys/kern/sys_generic.c:678
#13 0xc0baec15 in syscall  (frame=0xe7d2ad38) at /usr/src/sys/i386/i386/trap.c:1073
#14  0xc0b91b10 in Xint0x80_syscall () at  /usr/src/sys/i386/i386/exception.s:261
#15 0x00000033 in ?? ()
Previous  frame inner to this frame (corrupt stack?)

(kgdb)info threads
     at /usr/src/sys/kern/sched_ule.c:1864
  11 Thread 100004 (PID=11:  idle/idle: cpu0)  sched_switch (td=0xc5546900, newtd=0xc70e8480,  flags=1548)
    at /usr/src/sys/kern/sched_ule.c:1864
  10 Thread  100003 (PID=11: idle/idle: cpu1)  sched_switch (td=0xc5546b40,  newtd=0xc59c9480, flags=-972826840)
    at  /usr/src/sys/kern/sched_ule.c:1864
  9 Thread 100002 (PID=1: init)   sched_switch (td=0xc5546d80, newtd=0xc5546b40, flags=260)
    at  /usr/src/sys/kern/sched_ule.c:1864
  8 Thread 100001 (PID=10: audit)   sched_switch (td=0xc5548000, newtd=0xc55cc480, flags=260)
    at  /usr/src/sys/kern/sched_ule.c:1864
 (kgdb)thread 10
      [Switching to thread 10 (Thread 100003)]#0  sched_switch (td=0xc5546b40,  newtd=0xc59c9480, flags=-972826840)
      at  /usr/src/sys/kern/sched_ule.c:1864
     1864                    cpuid  = PCPU_GET(cpuid);
 (kgdb) bt
#0  sched_switch  (td=0xc5546b40, newtd=0xc59c9480, flags=-972826840) at  /usr/src/sys/kern/sched_ule.c:1864
#1  0xc603d9f0 in ?? ()
#2   0x00000002 in ?? ()
#3  0xc5e3e4a4 in __zt_transmit_chunk () from  /boot/kernel.old/./zaptel.ko
Previous frame inner to this frame  (corrupt stack?)
/*  --------------------------------------------------dump message End  ----------------------------------------------*/

sched_switch  (td=0xc5546b40, newtd=0xc59c9480, flags=-972826840)

The value of  flags is -972826840.This must be wrong.

In my driver module  (tej21.ko) will call __zt_transmit_chunk() function (in the module  zaptel.ko).

------------------ Original ------------------
From:  "John Baldwin"<jhb at freebsd.org>;
Date:  Thu, Jun 24, 2010 10:02 PM
To:  "freebsd-drivers"<freebsd-drivers at freebsd.org>; 
Cc:  "KGB"<380008156 at qq.com>; 
Subject:  Re: Kernel panic of "spin lock held too long"

 On Thursday 24 June 2010 7:30:00 am KGB wrote:
> Hi All:
>      I am writing a driver module (named tej21.ko) under freebsd8-release(i386) and I had a problem of the kernel panic "spin lock held too long".
>  
>      When I kldload my driver module without smp support (kern.smp.disabled = 1),every thing is OK.
>      But when I support the smp ,I met the problem 'Kernel panic of "spin lock held too long" '.
>  
>      My Hardware : 
>           1.CPU: Pentium(R) Dual-Core  CPU      E5200  @ 2.50GHz (2499.95-MHz 686-class CPU)
>  
>      Software:
>           freeBSD 8 release i386.
>  
>      The dump message is below.Why is there this problem,can someone give me advice? 
>  
>       /*------------------------------dump message ----------------------------------------------*/
>       Unread portion of the kernel message buffer:
>    spin lock 0xc0d9e580 (sched lock 1) held by 0xc5546b40 (tid 100003) too long

Can you get a trace of this thread (100003)?

-- 
John Baldwin
_______________________________________________
freebsd-drivers at freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-drivers
To unsubscribe, send any mail to "freebsd-drivers-unsubscribe at freebsd.org"