kern/116308: kernel crash on 6.2-stable - mutex problem?

[Mohacsi Janos]

>Number:         116308
>Category:       kern
>Synopsis:       kernel crash on 6.2-stable - mutex problem?
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Wed Sep 12 14:40:01 GMT 2007
>Closed-Date:
>Last-Modified:
>Originator:     Mohacsi Janos
>Release:        FreeBSD 6.2-STABLE i386
>Organization:
NIIF
>Environment:
System: FreeBSD mignon.ki.iif.hu 6.2-STABLE FreeBSD 6.2-STABLE #10: Thu Jul 12 20:44:48 CEST 2007 root at mignon2.ki.iif.hu:/usr/obj/usr/src/sys/MIGNON2 i386


>Description:
	kernel crash after network outage.
Here is the backtrace of crashed kernel.


mohacsi at mignon> sudo kgdb kernel.debug /var/crash/vmcore.2 
[GDB will not be able to debug user-mode threads: /usr/lib/libthread_db.so: Unde
fined symbol "ps_pglobal_lookup"]
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-marcel-freebsd".

Unread portion of the kernel message buffer:
kernel trap 12 with interrupts disabled


Fatal trap 12: page fault while in kernel mode
cpuid = 0; apic id = 00
fault virtual address   = 0x104
fault code              = supervisor read, page not present
instruction pointer     = 0x20:0xc0554445
stack pointer           = 0x28:0xe35db978
frame pointer           = 0x28:0xe35db98c
code segment            = base 0x0, limit 0xfffff, type 0x1b
                        = DPL 0, pres 1, def32 1, gran 1
processor eflags        = resume, IOPL = 0
current process         = 15 (swi4: clock sio)
trap number             = 12
panic: page fault
cpuid = 0
Uptime: 9d2h33m41s
Dumping 1023 MB (2 chunks)
  chunk 0: 1MB (159 pages) ... ok
  chunk 1: 1023MB (261872 pages) 1007 991 975 959 943 927 911 895 879 863 847 83
1 815 799 783 767 751 735 719 703 687 671 655 639 623 607 591 575 559 543 527 51
1 495 479 463 447 431 415 399 383 367 351 335 319 303 287 271 255 239 223 207 19
1 175 159 143 127 111 95 79 63 47 31 15

#0  doadump () at pcpu.h:165
165             __asm __volatile("movl %%fs:0,%0" : "=r" (td));
(kgdb) bt
#0  doadump () at pcpu.h:165
#1  0xc055ecd2 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:409
#2  0xc055f083 in panic (fmt=0xc074bfab "%s")
    at /usr/src/sys/kern/kern_shutdown.c:565
#3  0xc071b811 in trap_fatal (frame=0xe35db938, eva=0)
    at /usr/src/sys/i386/i386/trap.c:837
#4  0xc071aef9 in trap (frame=
      {tf_fs = -1067253752, tf_es = -480444376, tf_ds = -994050008, tf_edi = -99
5115648, tf_esi = 4, tf_ebp = -480396916, tf_isp = -480396956, tf_ebx = -9863358
64, tf_edx = 6, tf_ecx = 3, tf_eax = 1, tf_trapno = 12, tf_err = 0, tf_eip = -10
68153787, tf_cs = 32, tf_eflags = 65538, tf_esp = -986335864, tf_ss = -988281688
}) at /usr/src/sys/i386/i386/trap.c:270
#5  0xc070435a in calltrap () at /usr/src/sys/i386/i386/exception.s:139
#6  0xc0554445 in _mtx_lock_sleep (m=0xc535b588, tid=3299851648, opts=0, 
    file=0x0, line=0) at /usr/src/sys/kern/kern_mutex.c:546
#7  0xc06380fb in nd6_output (ifp=0xc4c0dc00, origifp=0x1, m0=0xc57b9600, 
    dst=0xc4f66c9c, rt0=0xc535b5ac) at /usr/src/sys/netinet6/nd6.c:2008
#8  0xc0631346 in ip6_output (m0=0xe35dbb4c, opt=0x0, ro=0xe35dbb4c, flags=0, 
    im6o=0x0, ifpp=0x0, inp=0xc6078ec4)
    at /usr/src/sys/netinet6/ip6_output.c:994
#9  0xc060fc19 in tcp_output (tp=0xc6180570)
    at /usr/src/sys/netinet/tcp_output.c:1059
#10 0xc061673b in tcp_timer_rexmt (xtp=0xc6180570)
    at /usr/src/sys/netinet/tcp_timer.c:537
#11 0xc056dd49 in softclock (dummy=0x0) at /usr/src/sys/kern/kern_timeout.c:290
#12 0xc054507f in ithread_execute_handlers (p=0xc4afa218, ie=0xc4b54b80)
    at /usr/src/sys/kern/kern_intr.c:682
#13 0xc05451d4 in ithread_loop (arg=0xc4add8c0)
    at /usr/src/sys/kern/kern_intr.c:765
#14 0xc0543a4b in fork_exit (callout=0xc0545170 <ithread_loop>, arg=0x1, 
    frame=0x1) at /usr/src/sys/kern/kern_fork.c:821
#15 0xc07043bc in fork_trampoline () at /usr/src/sys/i386/i386/exception.s:208
(kgdb)
>How-To-Repeat:
I had two similar crashes that seems nd6_output related.....

if you need more information I can extract it from the saved kernelcores - all the 3 exists...

>Fix:
 no idea 


>Release-Note:
>Audit-Trail:
>Unformatted: