kern/137117: panic after spinloc held too long with FreeBSD 7.2-RELEASE-p2 on CPU: Intel(R) Atom(TM) CPU 330 @ 1.60GHz (1618.46-MHz K8-class CPU)

[Jörg Ruppe-Tanner]

>Number:         137117
>Category:       kern
>Synopsis:       panic after spinloc held too long with FreeBSD 7.2-RELEASE-p2 on CPU: Intel(R) Atom(TM) CPU  330   @ 1.60GHz (1618.46-MHz K8-class CPU)
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Sat Jul 25 10:10:01 UTC 2009
>Closed-Date:
>Last-Modified:
>Originator:     Jörg Ruppe-Tanner
>Release:        FreeBSD 7.2-RELEASE-p2 #0
>Organization:
jrtnet.ch
>Environment:
FreeBSD pluto.jrtnet.ch 7.2-RELEASE-p2 FreeBSD 7.2-RELEASE-p2 #0: Wed Jun 24 00:14:35 UTC 2009     root at amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC  amd64
>Description:
cat /var/crash/info.0
Dump header from device /dev/mirror/gm0b
  Architecture: amd64
  Architecture Version: 2
  Dump Length: 977420288B (932 MB)
  Blocksize: 512
  Dumptime: Thu Jul 16 09:44:01 2009
  Hostname: pluto.jrtnet.ch
  Magic: FreeBSD Kernel Dump
  Version String: FreeBSD 7.2-RELEASE-p2 #0: Wed Jun 24 00:14:35 UTC 2009
    root at amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC
  Panic String: spin lock held too long
  Dump Parity: 32247041
  Bounds: 0
  Dump Status: good

kgdb /boot/kernel/kernel.symbols /var/crash/vmcore.0
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "amd64-marcel-freebsd"...

Unread portion of the kernel message buffer:
ssppiinn  lloocckk  00xxffffffffffffffff8800bb33eedd4c00  ((sscchheedd  lloocckk  10))  hheelldd  bbyy  0x0fxffffffffff0f000010714333aa560e 0( t(itdi d 10100005040)5 )t otoo ol olnogng

panic: spin lock held too long
cpuid = 0
Uptime: 12d13h12m53s
Physical memory: 2025 MB
Dumping 932 MB: 917 901 885 869 853 837 821 805 789 773 757 741 725 709 693 677 661 645 629 613 597 581 565 549 533 517 501 485 469 453 437 421 405 389 373 357 341 325 309 293 277 261 245 229 213 197 181 165 149 133 117 101 85 69 53 37 21 5

Reading symbols from /boot/kernel/geom_mirror.ko...Reading symbols from /boot/kernel/geom_mirror.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/geom_mirror.ko
Reading symbols from /boot/kernel/zfs.ko...Reading symbols from /boot/kernel/zfs.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/zfs.ko
Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from /boot/kernel/opensolaris.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/opensolaris.ko
Reading symbols from /boot/kernel/ipfw.ko...Reading symbols from /boot/kernel/ipfw.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/ipfw.ko
Reading symbols from /boot/kernel/ipdivert.ko...Reading symbols from /boot/kernel/ipdivert.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/ipdivert.ko
Reading symbols from /boot/kernel/nullfs.ko...Reading symbols from /boot/kernel/nullfs.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/nullfs.ko
#0  doadump () at pcpu.h:195
195     pcpu.h: No such file or directory.
        in pcpu.h
(kgdb) bt
#0  doadump () at pcpu.h:195
#1  0x0000000000000004 in ?? ()
#2  0xffffffff8050df79 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:418
#3  0xffffffff8050e382 in panic (fmt=0x104 <Address 0x104 out of bounds>) at /usr/src/sys/kern/kern_shutdown.c:574
#4  0xffffffff8050110c in _mtx_lock_spin_failed (m=Variable "m" is not available.
) at /usr/src/sys/kern/kern_mutex.c:449
#5  0xffffffff805011ae in _mtx_lock_spin (m=0xffffffff80b3edc0, tid=18446742974218086112, opts=Variable "opts" is not available.
) at /usr/src/sys/kern/kern_mutex.c:482
#6  0xffffffff805321be in sched_idletd (dummy=Variable "dummy" is not available.
) at /usr/src/sys/kern/sched_ule.c:801
#7  0xffffffff804ea973 in fork_exit (callout=0xffffffff805320b0 <sched_idletd>, arg=0x0, frame=0xfffffffe80033c80) at /usr/src/sys/kern/kern_fork.c:810
#8  0xffffffff807b748e in fork_trampoline () at /usr/src/sys/amd64/amd64/exception.S:455
#9  0x0000000000000000 in ?? ()
#10 0x0000000000000000 in ?? ()
#11 0x0000000000000001 in ?? ()
#12 0x0000000000000000 in ?? ()
#13 0x0000000000000000 in ?? ()
#14 0x0000000000000000 in ?? ()
#15 0x0000000000000000 in ?? ()
#16 0x0000000000000000 in ?? ()
#17 0x0000000000000000 in ?? ()
#18 0x0000000000000000 in ?? ()
#19 0x0000000000000000 in ?? ()
#20 0x0000000000000000 in ?? ()
#21 0x0000000000000000 in ?? ()
#22 0x0000000000000000 in ?? ()
---Type <return> to continue, or q <return> to quit---q
Quit
(kgdb)          
>How-To-Repeat:
Still waiting until the system is over a longer time 100% idle.

Workaround is to produce a little load and interupts. 
I do gnu-watch "ntpq -p" and gnu-watch "vmstat 1 2" and this even remote on a another machine.
>Fix:


>Release-Note:
>Audit-Trail:
>Unformatted: