Panic on 6.0-RELEASE-p4 - Fatal trap 12

[Hugo Silva]

Hello list,

I've experienced a panic on a heavily loaded webserver running FreeBSD 
6.0-RELEASE-p4, just some minutes ago. Just incase it's a new problem, 
I'm posting it here.

# kgdb ./kernel.debug ./vmcore.0
[GDB will not be able to debug user-mode threads: 
/usr/lib/libthread_db.so: Undefined symbol "ps_pglobal_lookup"]
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain 
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-marcel-freebsd".

Unread portion of the kernel message buffer:


Fatal trap 12: page fault while in kernel mode
cpuid = 0; apic id = 00
fault virtual address   = 0xac
fault code              = supervisor write, page not present
instruction pointer     = 0x20:0x806675ea
stack pointer           = 0x28:0xe30d2c18
frame pointer           = 0x28:0xe30d2c44
code segment            = base 0x0, limit 0xfffff, type 0x1b
                        = DPL 0, pres 1, def32 1, gran 1
processor eflags        = interrupt enabled, resume, IOPL = 0
current process         = 86 (swi4: clock sio)
trap number             = 12
panic: page fault
cpuid = 0
Uptime: 49d16h55m25s
Dumping 2047 MB (3 chunks)
  chunk 0: 1MB (159 pages) ... ok
  chunk 1: 2046MB (523760 pages) 2030 2014 1998 1982 1966 1950 1934 1918 
1902 1886 1870 1854 1838 1822 1806 1790 1774 1758 1742 1726 1710 1694 
1678 1662 1646 1630 1614 1598 1582 1566 1550 1534 1518 1502 1486 1470 
1454 1438 1422 1406 1390 1374 1358 1342 1326 1310 1294 1278 1262 1246 
1230 1214 1198 1182 1166 1150 1134 1118 1102 1086 1070 1054 1038 1022 
1006 990 974 958 942 926 910 894 878 862 846 830 814 798 782 766 750 734 
718 702 686 670 654 638 622 606 590 574 558 542 526 510 494 478 462 446 
430 414 398 382 366 350 334 318 302 286 270 254 238 222 206 190 174 158 
142 126 110 94 78 62 46 30 14 ... ok
  chunk 2: 1MB (128 pages)

#0  doadump () at pcpu.h:165
165     pcpu.h: No such file or directory.
        in pcpu.h






(kgdb) backtrace
#0  doadump () at pcpu.h:165
#1  0x805aef82 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:399
#2  0x805af374 in panic (fmt=0x80777163 "%s") at 
/usr/src/sys/kern/kern_shutdown.c:555
#3  0x8075107c in trap_fatal (frame=0xe30d2bd8, eva=0) at 
/usr/src/sys/i386/i386/trap.c:831
#4  0x80750d2f in trap_pfault (frame=0xe30d2bd8, usermode=0, eva=172) at 
/usr/src/sys/i386/i386/trap.c:742
#5  0x807508bd in trap (frame=
      {tf_fs = 8, tf_es = 40, tf_ds = -2130444248, tf_edi = 0, tf_esi = 
4, tf_ebp = -485675964, tf_isp = -485676028, tf_ebx = -1970586144, 
tf_edx = -2130424216, tf_ecx = -2091403008, tf_eax = 4, tf_trapno = 12, 
tf_err = 2, tf_eip = -2140768790, tf_cs = 32, tf_eflags = 66195, tf_esp 
= -1970586144, tf_ss = 0})
    at /usr/src/sys/i386/i386/trap.c:432
#6  0x8073a2fa in calltrap () at /usr/src/sys/i386/i386/exception.s:139
#7  0x806675ea in tcp_timer_2msl_tw (reuse=0) at atomic.h:149
#8  0x80666f7c in tcp_slowtimo () at /usr/src/sys/netinet/tcp_timer.c:136
#9  0x805f3589 in pfslowtimo (arg=0x0) at 
/usr/src/sys/kern/uipc_domain.c:475
#10 0x805bec76 in softclock (dummy=0x0) at 
/usr/src/sys/kern/kern_timeout.c:290
#11 0x80592ea8 in ithread_loop (arg=0x83522e80) at 
/usr/src/sys/kern/kern_intr.c:547
#12 0x80591ae0 in fork_exit (callout=0x80592cf0 <ithread_loop>, arg=0x4, 
frame=0x4) at /usr/src/sys/kern/kern_fork.c:789
#13 0x8073a35c in fork_trampoline () at 
/usr/src/sys/i386/i386/exception.s:208


I've got a dozen more webservers (most on -p5, tho) that have been up 
for a couple of months now, and none experienced this. All this server 
does is running apache and ssh. The /tmp partition is mounted as md, if 
it matters.

If this seems interesting and more information is needed, let me know.