kern/111458: [panic] Panic on 6.2-RELEASE AMD in kern_mutex

Kris Kennaway kris at obsecurity.org
Thu Jun 14 08:16:14 UTC 2007


On Thu, Jun 14, 2007 at 10:08:08AM +0200, Remco Bressers wrote:
> >> > > > > > >  To update this bugreport and to keep it 'warm'
> >> > > > > > >  
> >> > > > > > >  I've got the very same problem overhere.
> >> > > > > > >  
> >> > > > > > >  Our box : 
> >> > > > > > >  
> >> > > > > > >  # uname -r
> >> > > > > > >  6.2-RELEASE-p2
> >> > > > > > >  
> >> > > > > > >  This is an amd64 release
> >> > > > > > >  
> >> > > > > > >  Updates on GENERIC : 
> >> > > > > > >  
> >> > > > > > >  options         QUOTA 
> >> > > > > > >  device          pf
> >> > > > > > >  device          pflog
> >> > > > > > >  options		SMP
> >> > > > > > >  
> >> > > > > > >  Kernel messages : 
> >> > > > > > >  
> >> > > > > > >  cpuid = 0; apic id = 00
> >> > > > > > >  fault virtual address = 0x18c
> >> > > > > > >  fault code = supervisor read, page not present
> >> > > > > > >  current process = 5 (thread taskq)
> >> > > > > > >  trap number = 12
> >> > > > > > >  panic: page fault
> >> > > > > > >  cpuid = 0
> >> > > > > > 
> >> > > > > > How do you know it is "the very same problem"?  In order to 
> >determine
> >> > > > > > this you need to compare backtraces from the panic, which you 
> >didn't
> >> > > > > > provide.
> >> > > > > 
> >> > > > > I contacted the submitter for this problem and compared hardware 
> >and
> >> > > > > software. The symptoms are the same
> >> > > > 
> >> > > > Which just means "it crashed"
> >> > > > 
> >> > > > , the kernel panic is the same and
> >> > > > 
> >> > > > The panic message also just means "it crashed"
> >> > > > 
> >> > > > > the installed FreeBSD version is exactly the same.
> >> > > > 
> >> > > > You're both running the most recent version, no real surprises 
> >there.
> >> > > > 
> >> > > > > Sounds fair enough to me.  Ofcourse i cannot be 100% sure, but it
> >> > > > > sounds too obvious to me.
> >> > > > 
> >> > > > Well, maybe, but I respectfully submit that you don't understand 
> >the
> >> > > > issue well enough to conclude that :) Please follow up with a
> >> > > > backtrace and then we'll see where things stand.
> >> > > 
> >> > > You didn't write anything about the fact that software combinations +
> >> > > hardware is almost identical! That's no coincidence in my humble
> >> > > opinion :). Anyway..
> >> > 
> >> > I'd prefer not to prematurely jump to conclusions before you have any
> >> > supporting evidence.  It is of course an obviously true statement that
> >> > two identical systems may panic in two completely different and
> >> > unrelated ways.
> >> > 
> >> > > The problem in this case is, that the backtrace isn't written to 
> >disk. I
> >> > > must wait for the next opportunity to get that backtrace.
> >> > 
> >> > OK, let us know.
> 
> Hi Kris,
> 
> The crash just happened again and there's a dump in /var/crash now and
> it's 2GB big :
> 
> joule# ls -l
> total 2059510
> -rw-r--r--  1 root  wheel           2 May  7 11:06 bounds
> -rw-------  1 root  wheel         445 May  7 11:06 info.0
> -rw-r--r--  1 root  wheel           5 Jan 12 08:13 minfree
> -rw-------  1 root  wheel  2146025472 May  7 11:07 vmcore.0
> 
> 
> 
> I got the following output from kgdb :
> 
> joule# kgdb kernel.debug /var/crash/vmcore.0
> [GDB will not be able to debug user-mode
> threads: /usr/lib/libthread_db.so: Undefined symbol "ps_pglobal_lookup"]
> GNU gdb 6.1.1 [FreeBSD]
> Copyright 2004 Free Software Foundation, Inc.
> GDB is free software, covered by the GNU General Public License, and you
> are
> welcome to change it and/or distribute copies of it under certain
> conditions.
> Type "show copying" to see the conditions.
> There is absolutely no warranty for GDB.  Type "show warranty" for
> details.
> This GDB was configured as "amd64-marcel-freebsd".
> 
> Unread portion of the kernel message buffer:
> panic: page fault
> cpuid = 0
> Uptime: 12d10h56m24s
> Dumping 2046 MB (2 chunks)
>   chunk 0: 1MB (155 pages) ... ok
>   chunk 1: 2046MB (523776 pages) 2031 2015 1999 1983 1967 1951 1935 1919
> 1903 1887 1871 1855 1839 1823 1807 1791 1775 1759 1743 1727 1711 1695
> 1679 1663 1647 1631 1615 1599 1583 1567 1551 1535 1519 1503 1487 1471
> 1455 1439 1423 1407 1391 1375 1359 1343 1327 1311 1295 1279 1263 1247
> 1231 1215 1199 1183 1167 1151 1135 1119 1103 1087 1071 1055 1039 1023
> 1007 991 975 959 943 927 911 895 879 863 847 831 815 799 783 767 751 735
> 719 703 687 671 655 639 623 607 591 575 559 543 527 511 495 479 463 447
> 431 415 399 383 367 351 335 319 303 287 271 255 239 223 207 191 175 159
> 143 127 111 95 79 63 47 31 15
> 
> #0  doadump () at pcpu.h:172
> 172             __asm __volatile("movq %%gs:0,%0" : "=r" (td));
> 
> (kgdb) backtrace
> 
> #0  doadump () at pcpu.h:172
> #1  0x0000000000000004 in ?? ()
> #2  0xffffffff8021f787 in boot (howto=260)
> at /usr/src/sys/kern/kern_shutdown.c:409
> #3  0xffffffff8021fe21 in panic (fmt=0xffffff007b950720 "")
> at /usr/src/sys/kern/kern_shutdown.c:565
> #4  0xffffffff8035edcf in trap_fatal (frame=0xffffff007b950720,
> eva=18446742976271278080) at /usr/src/sys/amd64/amd64/trap.c:660
> #5  0xffffffff8035f2f6 in trap (frame=
>       {tf_rdi = 24, tf_rsi = -1097438263520, tf_rdx = 6, tf_rcx =
> 3221225730, tf_r8 = -1315361520, tf_r9 = -1097438492872, tf_rax = 1,
> tf_rbx = -1097589680120, tf_rbp = 4, tf_r10 = -2142164360, tf_r11 = 0,
> tf_r12 = -1097438263520, tf_r13 = 4, tf_r14 = 1, tf_r15 = 20, tf_trapno
> = 12, tf_addr = 396, tf_flags = -1097589680120, tf_err = 0, tf_rip =
> -2145299289, tf_cs = 8, tf_rflags = 65538, tf_rsp = -1315361936, tf_ss =
> 16}) at /usr/src/sys/amd64/amd64/trap.c:238
> #6  0xffffffff8034ac1b in calltrap ()
> at /usr/src/sys/amd64/amd64/exception.S:168
> #7  0xffffffff802154a7 in _mtx_lock_sleep (m=0xffffff00728e9808,
> tid=18446742976271288096, opts=6, file=0xc0000102 <Address 0xc0000102
> out of bounds>,
>     line=-1315361520) at /usr/src/sys/kern/kern_mutex.c:546
> #8  0xffffffff8027256d in unp_gc (arg=0x18, pending=2073364256)
> at /usr/src/sys/kern/uipc_usrreq.c:1714
> #9  0xffffffff80245d65 in taskqueue_run (queue=0xffffff0000792c00)
> at /usr/src/sys/kern/subr_taskqueue.c:257
> #10 0xffffffff80246ab5 in taskqueue_thread_loop (arg=0x18)
> at /usr/src/sys/kern/subr_taskqueue.c:376
> #11 0xffffffff80206c47 in fork_exit (callout=0xffffffff80246a30
> <taskqueue_thread_loop>, arg=0xffffffff8051a0b0,
> frame=0xffffffffb1992c50)
>     at /usr/src/sys/kern/kern_fork.c:821
> #12 0xffffffff8034af7e in fork_trampoline ()
> at /usr/src/sys/amd64/amd64/exception.S:394
> #13 0x0000000000000000 in ?? ()
> #14 0x0000000000000000 in ?? ()
> #15 0x0000000000000001 in ?? ()
> #16 0x0000000000000000 in ?? ()
> #17 0x0000000000000000 in ?? ()
> #18 0x0000000000000000 in ?? ()
> #19 0x0000000000000000 in ?? ()
> #20 0x0000000000000000 in ?? ()
> #21 0x0000000000000000 in ?? ()
> #22 0x0000000000000000 in ?? ()
> #23 0x0000000000000000 in ?? ()
> #24 0x0000000000000000 in ?? ()
> #25 0x0000000000000000 in ?? ()
> #26 0x0000000000000000 in ?? ()
> #27 0x0000000000000000 in ?? ()
> #28 0x0000000000000000 in ?? ()
> #29 0x0000000000000000 in ?? ()
> #30 0x0000000000000000 in ?? ()
> #31 0x0000000000000000 in ?? ()
> #32 0x0000000000000000 in ?? ()
> #33 0x0000000000000000 in ?? ()
> #34 0x0000000000000000 in ?? ()
> #35 0x0000000000000000 in ?? ()
> #36 0x0000000000000000 in ?? ()
> #37 0x0000000000000000 in ?? ()
> #38 0x0000000000000000 in ?? ()
> #39 0x0000000000000000 in ?? ()
> #40 0x0000000000000000 in ?? ()
> #41 0x0000000000000000 in ?? ()
> #42 0x0000000000000000 in ?? ()
> #43 0x0000000000000000 in ?? ()
> #44 0x0000000000000000 in ?? ()
> #45 0x00000000006d6000 in ?? ()
> #46 0xffffff00728e9808 in ?? ()
> #47 0xffffffff8051a920 in turnstile_chains ()
> #48 0x0000000000000001 in ?? ()
> #49 0xffffff007b94e000 in ?? ()
> #50 0xffffff007b950be0 in ?? ()
> #51 0xffffffffb1992aa8 in ?? ()
> #52 0xffffff007b950720 in ?? ()
> #53 0xffffffff80234cc6 in sched_switch (td=0xffffffff8051a0b0,
> newtd=0x0, flags=0) at /usr/src/sys/kern/sched_4bsd.c:973
> #54 0x0000000000000000 in ?? ()
> #55 0x0000000000000000 in ?? ()
> #56 0x0000000000000000 in ?? ()
> #57 0x0000000000000000 in ?? ()
> #58 0x0000000000000000 in ?? ()
> #59 0x0000000000000000 in ?? ()
> #60 0x0000000000000000 in ?? ()
> #61 0x0000000000000000 in ?? ()
> #62 0x0000000000000000 in ?? ()
> #63 0x0000000000000000 in ?? ()
> #64 0x0000000000000000 in ?? ()
> #65 0x0000000000000000 in ?? ()
> #66 0x0000000000000000 in ?? ()
> #67 0x0000000000000000 in ?? ()
> #68 0x0000000000000000 in ?? ()
> #69 0x0000000000000000 in ?? ()
> #70 0x0000000000000000 in ?? ()
> #71 0x0000000000000000 in ?? ()
> #72 0x0000000000000000 in ?? ()
> #73 0x0000000000000000 in ?? ()
> #74 0x0000000000000000 in ?? ()
> #75 0x0000000000000000 in ?? ()
> #76 0x0000000000000000 in ?? ()
> #77 0x0000000000000000 in ?? ()
> #78 0x0000000000000000 in ?? ()
> #79 0x0000000000000000 in ?? ()
> #80 0x0000000000000000 in ?? ()
> #81 0x0000000000000000 in ?? ()
> #82 0x0000000000000000 in ?? ()
> #83 0x0000000000000000 in ?? ()
> #84 0x0000000000000000 in ?? ()
> #85 0x0000000000000000 in ?? ()
> #86 0x0000000000000000 in ?? ()
> #87 0x0000000000000000 in ?? ()
> #88 0x0000000000000000 in ?? ()
> #89 0x0000000000000000 in ?? ()
> #90 0x0000000000000000 in ?? ()
> #91 0x0000000000000000 in ?? ()
> #92 0x0000000000000000 in ?? ()
> #93 0x0000000000000000 in ?? ()
> #94 0x0000000000000000 in ?? ()
> #95 0x0000000000000000 in ?? ()
> #96 0x0000000000000000 in ?? ()
> #97 0x0000000000000000 in ?? ()
> #98 0x0000000000000000 in ?? ()
> #99 0x0000000000000000 in ?? ()
> #100 0x0000000000000000 in ?? ()
> #101 0x0000000000000000 in ?? ()
> #102 0x0000000000000000 in ?? ()
> #103 0x0000000000000000 in ?? ()
> #104 0x0000000000000000 in ?? ()
> #105 0x0000000000000000 in ?? ()
> #106 0x0000000000000000 in ?? ()
> #107 0x0000000000000000 in ?? ()
> #108 0x0000000000000000 in ?? ()
> #109 0x0000000000000000 in ?? ()
> #110 0x0000000000000000 in ?? ()
> #111 0x0000000000000000 in ?? ()
> #112 0x0000000000000000 in ?? ()
> #113 0x0000000000000000 in ?? ()
> #114 0x0000000000000000 in ?? ()
> #115 0x0000000000000000 in ?? ()
> #116 0x0000000000000000 in ?? ()
> #117 0x0000000000000000 in ?? ()
> #118 0x0000000000000000 in ?? ()
> #119 0x0000000000000000 in ?? ()
> #120 0x0000000000000000 in ?? ()
> #121 0x0000000000000000 in ?? ()
> #122 0x0000000000000000 in ?? ()
> #123 0x0000000000000000 in ?? ()
> #124 0x0000000000000000 in ?? ()
> #125 0x0000000000000000 in ?? ()
> Cannot access memory at address 0xffffffffb1993000
> 
> 
> I hope this will be useful information.
> 
> 
> Regards,
> 
> Remco Bressers
> Signet B.V.
> 
> 
> -----------------
> 
> Hi,
> 
> Is there anything new on this case yet?

Adding rwatson to the CC since he may be interested in it.

Kris


More information about the freebsd-bugs mailing list