kern/121955: freebsd 7.0 panic with mpd
Alexander V. Shulikov
shulikov at gmail.com
Sat Mar 22 10:10:01 UTC 2008
>Number: 121955
>Category: kern
>Synopsis: freebsd 7.0 panic with mpd
>Confidential: no
>Severity: critical
>Priority: high
>Responsible: freebsd-bugs
>State: open
>Quarter:
>Keywords:
>Date-Required:
>Class: sw-bug
>Submitter-Id: current-users
>Arrival-Date: Sat Mar 22 10:10:01 UTC 2008
>Closed-Date:
>Last-Modified:
>Originator: Alexander V. Shulikov
>Release: 7.0-RELEASE
>Organization:
ISP DonEC
>Environment:
FreeBSD hostname 7.0-RELEASE FreeBSD 7.0-RELEASE #0: Sat Mar 22 10:47:30 EET 2008 hostname:/usr/obj/usr/src/sys/kernconf i386
>Description:
I use FreeBSD server as pptp server with mpd-5.0. Before 7.0 it work under FreeBSD 6.2+mpd-4.4. When I try update to FreeBSD 6.3 I received systems reboot in different intervals of time and no crash dumps.
Then I try configure new server on FreeBSD 7.0. Kernel was build with GENERIC config with turned of options COMPAT_FREEBSD5 and 6 and added IPFW, DUMMYNET, option HZ=1000 and some drivers with nodevice.
options IPFIREWALL #firewall
options IPFIREWALL_DEFAULT_TO_ACCEPT #allow everything by default
options IPFIREWALL_FORWARD #packet destination changes
options IPDIVERT #divert sockets
options DUMMYNET
options HZ=1000
With this kernel system works. But when I turned on mpd-5.0 and clients connecting to them after some time (3 min - 30 min) system or freeze with message:
Fault double fault:
eip = ..
esp = ..
or panic with crash dump.
# cat info.0
Dump header from device /dev/ad4s1b
Architecture: i386
Architecture Version: 2
Dump Length: 65822720B (62 MB)
Blocksize: 512
Dumptime: Sat Mar 22 08:13:28 2008
Hostname: uzbek.matrixhome.net
Magic: FreeBSD Kernel Dump
Version String: FreeBSD 7.0-RELEASE #0: Fri Mar 21 22:08:21 EET 2008
hostname:/usr/obj/usr/src/sys/kernconf
Panic String: double fault
Dump Parity: 3717215795
Bounds: 0
Dump Status: good
# kgdb /root/debug/kernel.debug /var/crash/vmcore.0
[GDB will not be able to debug user-mode threads: /usr/lib/libthread_db.so: Undefined symbol "ps_pglobal_lookup"]
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i386-marcel-freebsd".
Unread portion of the kernel message buffer:
Fatal double fault:
eip = 0xc40914f5
esp = 0xe3ff4000
ebp = 0xe3ff40b4
cpuid = 0; apic id = 00
panic: double fault
cpuid = 0
Uptime: 3m43s
Physical memory: 1015 MB
Dumping 62 MB: 47 31 15
#0 doadump () at pcpu.h:195
195 pcpu.h: No such file or directory.
in pcpu.h
(kgdb) backtrace
#0 doadump () at pcpu.h:195
#1 0xc05591c7 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:409
#2 0xc0559489 in panic (fmt=Variable "fmt" is not available.
) at /usr/src/sys/kern/kern_shutdown.c:563
#3 0xc079007b in dblfault_handler () at /usr/src/sys/i386/i386/trap.c:928
#4 0xc40914f5 in ?? ()
#5 0xc4238980 in ?? ()
#6 0xc40bc7e0 in ?? ()
.....there is many addresses, that I can't find in *.debug with addr2line
#439 0x00000000 in ?? ()
#440 0xe3ff4a14 in ?? ()
#441 0xe3ff49fc in ?? ()
#442 0xc061495e in ipfw_chk (args=0xc4238ad4) at /usr/src/sys/netinet/ip_fw2.c:2659
Previous frame inner to this frame (corrupt stack?)
Then I rebuild kernel added:
options NETGRAPH # netgraph(4) system
options NETGRAPH_DEBUG # enable extra debugging, this
# affects netgraph(4) and nodes
# Node types
options NETGRAPH_CAR
options NETGRAPH_IFACE
options NETGRAPH_KSOCKET
options NETGRAPH_NETFLOW
options NETGRAPH_PPP
options NETGRAPH_PPTPGRE
options NETGRAPH_SOCKET
options NETGRAPH_TCPMSS
options NETGRAPH_TEE
options NETGRAPH_VJC
After this kldstat:
# kldstat
Id Refs Address Size Name
1 10 0xc0400000 533f50 kernel
2 1 0xc0934000 6a32c acpi.ko
3 1 0xc3fcd000 3000 pflog.ko
4 1 0xc3fd0000 33000 pf.ko
5 1 0xc40dc000 4000 ng_mppc.ko
6 1 0xc40e0000 2000 rc4.ko
7 1 0xc415d000 2000 blank_saver.ko
And panic was:
# cat /var/crash/info.1
Dump header from device /dev/ad4s1b
Architecture: i386
Architecture Version: 2
Dump Length: 61734912B (58 MB)
Blocksize: 512
Dumptime: Sat Mar 22 11:17:16 2008
Hostname: uzbek.matrixhome.net
Magic: FreeBSD Kernel Dump
Version String: FreeBSD 7.0-RELEASE #0: Sat Mar 22 10:47:30 EET 2008
hostname:/usr/obj/usr/src/sys/kernconf
Panic String: double fault
Dump Parity: 3313853732
Bounds: 1
Dump Status: good
# kgdb /usr/obj/usr/src/sys/uzbek/kernel.debug /var/crash/vmcore.1
[GDB will not be able to debug user-mode threads: /usr/lib/libthread_db.so: Undefined symbol "ps_pglobal_lookup"]
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i386-marcel-freebsd".
Unread portion of the kernel message buffer:
Fatal double fault:
eip = 0xc42bbdfb
esp = 0xe3fedff4
ebp = 0xe3fee030
cpuid = 0; apic id = 00
panic: double fault
cpuid = 0
Uptime: 7m16s
Physical memory: 1015 MB
Dumping 58 MB: (CTRL-C to abort) 43 (CTRL-C to abort) 27 (CTRL-C to abort) 11
#0 doadump () at pcpu.h:195
195 pcpu.h: No such file or directory.
in pcpu.h
(kgdb) backtrace
#0 doadump () at pcpu.h:195
#1 0xc055a0b7 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:409
#2 0xc055a379 in panic (fmt=Variable "fmt" is not available.
) at /usr/src/sys/kern/kern_shutdown.c:563
#3 0xc07ac2ab in dblfault_handler () at /usr/src/sys/i386/i386/trap.c:928
#4 0xc42bbdfb in ?? ()
Cannot access memory at address 0xe3fedff4
(kgdb)
Some notes:
Server used for connecting 100-150 tunnels at one time. In system used ipfw and pf. ipfw for dummynet and count with net.inet.ip.fw.one_pass=0
pf for filtering, nat and scrub
The same system on FreeBSD 6.2-RELEASE-p11 (mpd-5.0, ipfw. pf) don't panic after 3 minutes.
I can't update my server from 6.2, because in this configuration I have troubles in 6.3 and in 7.0. System or freezes or panic and reboot.
>How-To-Repeat:
Read full description.
>Fix:
>Release-Note:
>Audit-Trail:
>Unformatted:
More information about the freebsd-bugs
mailing list