kern/146719: PF or dumynet kernel panic

Tue May 18 23:00:10 UTC 2010

>Number:         146719
>Category:       kern
>Synopsis:       PF or dumynet kernel panic
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Tue May 18 23:00:09 UTC 2010
>Closed-Date:
>Last-Modified:
>Originator:     Alexander
>Release:        FreeBSD 7.3-REALESE i386
>Organization:
>Environment:
FreeBSD server 7.3-RELEASE FreeBSD 7.3-RELEASE #0: Sat May 15 14:34:54 EEST 2010     xxx at xxx.xxx:/usr/src/sys/i386/compile/MYKERN  i386

Hardware:
CPU: Pentium(R) Dual-Core  CPU      E6300  @ 2.80GHz (2799.98-MHz 686-class CPU)
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
 cpu0 (BSP): APIC ID:  0
 cpu1 (AP): APIC ID:  1
rlphy0: <RTL8201L 10/100 media interface> PHY 1 on miibus0
rlphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
re0: Ethernet address: 00:30:67:1b:ef:b0
re0: [FILTER]
fxp0: <Intel 82559 Pro/100 Ethernet> port 0xec00-0xec3f mem 0xfebff000-0xfebfffff,0xfea00000-0xfeafffff irq 16 at device 1.0 on pci3
miibus1: <MII bus> on fxp0
inphy0: <i82555 10/100 media interface> PHY 1 on miibus1
inphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
fxp0: Ethernet address: 00:02:b3:0c:5c:1b
fxp0: [ITHREAD]
fxp1: <Intel 82559 Pro/100 Ethernet> port 0xe880-0xe8bf mem 0xfebfe000-0xfebfefff,0xfe800000-0xfe8fffff irq 17 at device 2.0 on pci3
miibus2: <MII bus> on fxp1
inphy1: <i82555 10/100 media interface> PHY 1 on miibus2
inphy1:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
fxp1: Ethernet address: 00:d0:b7:c3:ec:2f
fxp1: [ITHREAD]

>Description:
This machine is used as a Internet gateway with FW+Shaping (ipfw), NAT (pf), PPTP server (mpd5), freeradius, apache, mysql

Sometimes (1 crash per 1 or 2 days or even more) machine panics with:

#kgdb -c /var/crash/vmcore.7 /boot/kernel/
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-marcel-freebsd"...

Unread portion of the kernel message buffer:

Fatal trap 12: page fault while in kernel mode
cpuid = 0; apic id = 00
fault virtual address   = 0xbe
fault code              = supervisor read, page not present
instruction pointer     = 0x20:0xc313e140
stack pointer           = 0x28:0xd426aa8c
frame pointer           = 0x28:0xd426ab80
code segment            = base 0x0, limit 0xfffff, type 0x1b
                        = DPL 0, pres 1, def32 1, gran 1
processor eflags        = interrupt enabled, resume, IOPL = 0
current process         = 42 (dummynet)
trap number             = 12
panic: page fault
cpuid = 0
Uptime: 2d7h30m14s
Physical memory: 469 MB
Dumping 130 MB: 115 99 83 67 51 35 19 3

Reading symbols from /boot/kernel/acpi.ko...Reading symbols from /boot/kernel/acpi.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/acpi.ko
Reading symbols from /boot/kernel/pf.ko...Reading symbols from /boot/kernel/pf.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/pf.ko
Reading symbols from /boot/kernel/linux.ko...Reading symbols from /boot/kernel/linux.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/linux.ko
Reading symbols from /boot/kernel/ng_pppoe.ko...Reading symbols from /boot/kernel/ng_pppoe.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/ng_pppoe.ko
Reading symbols from /boot/kernel/ng_tee.ko...Reading symbols from /boot/kernel/ng_tee.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/ng_tee.ko
Reading symbols from /boot/kernel/ng_pptpgre.ko...Reading symbols from /boot/kernel/ng_pptpgre.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/ng_pptpgre.ko
Reading symbols from /boot/kernel/ng_ksocket.ko...Reading symbols from /boot/kernel/ng_ksocket.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/ng_ksocket.ko
Reading symbols from /boot/kernel/ng_iface.ko...Reading symbols from /boot/kernel/ng_iface.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/ng_iface.ko
Reading symbols from /boot/kernel/ng_ppp.ko...Reading symbols from /boot/kernel/ng_ppp.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/ng_ppp.ko
Reading symbols from /boot/kernel/ng_tcpmss.ko...Reading symbols from /boot/kernel/ng_tcpmss.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/ng_tcpmss.ko
#0  doadump () at pcpu.h:196
196     pcpu.h: No such file or directory.
        in pcpu.h

(kgdb) list *0xc313e140
0xc313e140 is in pf_test (/usr/src/sys/modules/pf/../../contrib/pf/net/pf.c:6969).
6964    #endif
6965                    DPFPRINTF(PF_DEBUG_URGENT,
6966                        ("pf_test: kif == NULL, if_xname %s\n", ifp->if_xname));
6967                    return (PF_DROP);
6968            }
6969            if (kif->pfik_flags & PFI_IFLAG_SKIP) {
6970    #ifdef __FreeBSD__
6971                    PF_UNLOCK();
6972    #endif
6973                    return (PF_PASS);

(kgdb) backtrace
#0  doadump () at pcpu.h:196
#1  0xc065f287 in boot (howto=260) at ../../../kern/kern_shutdown.c:418
#2  0xc065f559 in panic (fmt=Variable "fmt" is not available.
) at ../../../kern/kern_shutdown.c:574
#3  0xc08c247c in trap_fatal (frame=0xd426aa4c, eva=190) at ../../../i386/i386/trap.c:950
#4  0xc08c26e0 in trap_pfault (frame=0xd426aa4c, usermode=0, eva=190)
    at ../../../i386/i386/trap.c:863
#5  0xc08c3099 in trap (frame=0xd426aa4c) at ../../../i386/i386/trap.c:541
#6  0xc08a6d9b in calltrap () at ../../../i386/i386/exception.s:166
#7  0xc313e140 in pf_test (dir=1, ifp=0xc3a6b400, m0=0xd426abc4, eh=0x0, inp=0x0)
    at /usr/src/sys/modules/pf/../../contrib/pf/net/pf.c:6969
#8  0xc31447b9 in pf_check_in (arg=0x0, m=0xd426abc4, ifp=0xc3a6b400, dir=1, inp=0x0)
    at /usr/src/sys/modules/pf/../../contrib/pf/net/pf_ioctl.c:3647
#9  0xc070a578 in pfil_run_hooks (ph=0xc0a1ada0, mp=0xd426ac1c, ifp=0xc3a6b400, dir=1, inp=0x0)
    at ../../../net/pfil.c:78
#10 0xc07379c1 in ip_input (m=0xc3dd3000) at ../../../netinet/ip_input.c:416
#11 0xc0708d15 in netisr_dispatch (num=2, m=0xc3dd3000) at ../../../net/netisr.c:185
#12 0xc0729212 in dummynet_send (m=0xc3dd3000) at ../../../netinet/ip_dummynet.c:901
#13 0xc072a786 in dummynet_task (context=0x0, pending=1) at ../../../netinet/ip_dummynet.c:877
#14 0xc0696475 in taskqueue_run (queue=0xc2e73100) at ../../../kern/subr_taskqueue.c:282
#15 0xc0696688 in taskqueue_thread_loop (arg=0xc0a1a5a8) at ../../../kern/subr_taskqueue.c:401
#16 0xc06388e9 in fork_exit (callout=0xc06965c0 <taskqueue_thread_loop>, arg=0xc0a1a5a8,
    frame=0xd426ad38) at ../../../kern/kern_fork.c:811
#17 0xc08a6e10 in fork_trampoline () at ../../../i386/i386/exception.s:271

>How-To-Repeat:

>Fix:

>Release-Note:
>Audit-Trail:
>Unformatted: