kern/134584: spin lock held too long

Sat May 16 18:50:03 UTC 2009

>Number:         134584
>Category:       kern
>Synopsis:       spin lock held too long
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Sat May 16 18:50:02 UTC 2009
>Closed-Date:
>Last-Modified:
>Originator:     Barbara
>Release:        7-STABLE
>Organization:
>Environment:
FreeBSD satanasso.local.net 7.2-STABLE FreeBSD 7.2-STABLE #0: Mon May  4 23:00:17 CEST 2009     root at satanasso.local.net:/usr/obj/usr/src/sys/SATANASSO  i386

>Description:
Recently I'm sometimes getting (innocuous?) panic on shutdown.

# kgdb kernel.debug /var/crash/vmcore.11

GNU gdb 6.1.1 [FreeBSD]

GDB is free software, covered by the GNU General Public License, and you are

welcome to change it and/or distribute copies of it under certain conditions.

Type "show copying" to see the conditions.

There is absolutely no warranty for GDB.  Type "show warranty" for details.

This GDB was configured as "i386-marcel-freebsd"...

Unread portion of the kernel message buffer:

<118>.

<118>Terminated

<118>.

<118>May 16 18:34:57 satanasso syslogd: exiting on signal 15

Waiting (max 60 seconds) for system process `vnlru' to stop...done

Waiting (max 60 seconds) for system process `bufdaemon' to stop...done

Waiting (max 60 seconds) for system process `syncer' to stop...

Syncing disks, vnodes remaining...8 3 3 3 0 0 done

All buffers synced.

Uptime: 16h22m16s

Rebooting...

cpu_reset: Stopping other CPUs

spin lock 0xc08a7e80 (sched lock 1) held by 0xc508b460 (tid 100005) too long

panic: spin lock held too long

cpuid = 0

Uptime: 16h22m18s

Physical memory: 2039 MB

Dumping 171 MB: 156 140 124 108 92 76 60 44 28 12

Reading symbols from /boot/kernel/linux.ko...Reading symbols from /boot/kernel/linux.ko.symbols...done.

done.

Loaded symbols for /boot/kernel/linux.ko

Reading symbols from /boot/kernel/acpi.ko...Reading symbols from /boot/kernel/acpi.ko.symbols...done.

done.

Loaded symbols for /boot/kernel/acpi.ko

Reading symbols from /boot/kernel/linprocfs.ko...Reading symbols from /boot/kernel/linprocfs.ko.symbols...done.

done.

Loaded symbols for /boot/kernel/linprocfs.ko

Reading symbols from /boot/modules/nvidia.ko...done.

Loaded symbols for /boot/modules/nvidia.ko

Reading symbols from /boot/kernel/logo_saver.ko...Reading symbols from /boot/kernel/logo_saver.ko.symbols...done.

done.

Loaded symbols for /boot/kernel/logo_saver.ko

#0  doadump () at pcpu.h:196

196		__asm __volatile("movl %%fs:0,%0" : "=r" (td));

(kgdb) bt

#0  doadump () at pcpu.h:196

#1  0xc0597f17 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:418

#2  0xc05981ec in panic (fmt=Variable "fmt" is not available.

) at /usr/src/sys/kern/kern_shutdown.c:574

#3  0xc0589d6f in _mtx_lock_spin_failed (m=0x0) at /usr/src/sys/kern/kern_mutex.c:449

#4  0xc058a475 in _thread_lock_flags (td=0xc508b460, opts=0, file=0xc081a7e2 "/usr/src/sys/kern/kern_intr.c", line=758) at /usr/src/sys/kern/kern_mutex.c:537

#5  0xc0574d49 in intr_event_schedule_thread (ie=0xc5087a00) at /usr/src/sys/kern/kern_intr.c:758

#6  0xc0575ed9 in swi_sched (cookie=0xc507cd00, flags=Variable "flags" is not available.

) at /usr/src/sys/kern/kern_intr.c:990

#7  0xc055908f in hardclock (usermode=0, pc=3229481926) at /usr/src/sys/kern/kern_clock.c:339

#8  0xc07c8d5a in lapic_handle_timer (frame=0xc4caaba0) at /usr/src/sys/i386/i386/local_apic.c:705

#9  0xc07c126f in Xtimerint () at apic_vector.s:108

#10 0xc07dfbc6 in DELAY (n=1000000) at /usr/src/sys/i386/isa/clock.c:296

#11 0xc07dd6c3 in cpu_reset () at /usr/src/sys/i386/i386/vm_machdep.c:594

#12 0xc05975d3 in shutdown_reset (junk=0x0, howto=0) at /usr/src/sys/kern/kern_shutdown.c:493

#13 0xc0597f97 in boot (howto=0) at /usr/src/sys/kern/kern_shutdown.c:421

#14 0xc05980c7 in reboot (td=0xc508bd20, uap=0xc4caacfc) at /usr/src/sys/kern/kern_shutdown.c:172

#15 0xc07db295 in syscall (frame=0xc4caad38) at /usr/src/sys/i386/i386/trap.c:1090

#16 0xc07c0cf0 in Xint0x80_syscall () at /usr/src/sys/i386/i386/exception.s:255

#17 0x00000033 in ?? ()

Previous frame inner to this frame (corrupt stack?)

(kgdb) tid 100005

[Switching to thread 7 (Thread 100005)]#0  sched_switch (td=0xc508b460, newtd=Variable "newtd" is not available.

) at /usr/src/sys/kern/sched_ule.c:1944

1944			cpuid = PCPU_GET(cpuid);

(kgdb) list

1939			/*

1940			 * We may return from cpu_switch on a different cpu.  However,

1941			 * we always return with td_lock pointing to the current cpu's

1942			 * run queue lock.

1943			 */

1944			cpuid = PCPU_GET(cpuid);

1945			tdq = TDQ_CPU(cpuid);

1946	#ifdef	HWPMC_HOOKS

1947			if (PMC_PROC_IS_USING_PMCS(td->td_proc))

1948				PMC_SWITCH_CONTEXT(td, PMC_FN_CSW_IN);

(kgdb) bt

#0  sched_switch (td=0xc508b460, newtd=Variable "newtd" is not available.

) at /usr/src/sys/kern/sched_ule.c:1944

#1  0xc059fe83 in mi_switch (flags=Variable "flags" is not available.

) at /usr/src/sys/kern/kern_synch.c:444

#2  0xc07cf192 in ipi_bitmap_handler (frame=

      {tf_fs = 8, tf_es = 40, tf_ds = 40, tf_edi = -653390888, tf_esi = -989285280, tf_ebp = -993289100, tf_isp = -993289144, tf_ebx = -1064707952, tf_edx = -989283040, tf_ecx = -989283040, tf_eax = -989283040, tf_trapno = 0, tf_err = 0, tf_eip = -1067934059, tf_cs = 32, tf_eflags = 582, tf_esp = -1066211856, tf_ss = 0}) at /usr/src/sys/i386/i386/mp_machdep.c:1179

#3  0xc07c137e in Xipi_intr_bitmap_handler () at apic_vector.s:284

#4  0xc0589e95 in _mtx_lock_sleep (m=0xc089d890, tid=3305682016, opts=0, file=0x0, line=0) at /usr/src/sys/kern/kern_mutex.c:349

#5  0xc05aa64b in softclock (dummy=0x0) at /usr/src/sys/kern/kern_timeout.c:241

#6  0xc057609b in ithread_loop (arg=0xc50882d0) at /usr/src/sys/kern/kern_intr.c:1088

#7  0xc0572c07 in fork_exit (callout=0xc0575ee0 <ithread_loop>, arg=0xc50882d0, frame=0xc4cb9d38) at /usr/src/sys/kern/kern_fork.c:810

#8  0xc07c0d00 in fork_trampoline () at /usr/src/sys/i386/i386/exception.s:264

(kgdb) quit

>How-To-Repeat:

>Fix:

>Release-Note:
>Audit-Trail:
>Unformatted: