kern/123741: [netgraph] [panic] kernel panic due to netgraph mpd
C Fan
mimielliot at gmail.com
Fri May 16 20:20:01 UTC 2008
>Number: 123741
>Category: kern
>Synopsis: [netgraph] [panic] kernel panic due to netgraph mpd
>Confidential: no
>Severity: critical
>Priority: high
>Responsible: freebsd-bugs
>State: open
>Quarter:
>Keywords:
>Date-Required:
>Class: sw-bug
>Submitter-Id: current-users
>Arrival-Date: Fri May 16 20:20:00 UTC 2008
>Closed-Date:
>Last-Modified:
>Originator: C Fan
>Release: FreeBSD 7.0 stable
>Organization:
>Environment:
FreeBSD xxx 7.0-STABLE FreeBSD 7.0-STABLE #6: Sat May 10 05:27:18 PDT 2008 root at xxx:/usr/obj/usr/src/sys/XXX i386
>Description:
I'm having a system, console, and keyboard locked up problems on both of my production server and home server after upgrading to Freebsd 7.0. There is a mpd VPN setup between these 2 servers. A PR has been filed regarding to the problem.
http://www.freebsd.org/cgi/query-pr.cgi?pr=123729
After the upgrade, my home server always got locked up after few hours. I compiled my home server with DDB and KDB options. The problem seemed to be gone. However, after few days, I got a kernel panic and core dump.
Fatal trap 12: page fault while i kernel mode
cpuid = 0; apic id = 00
fault virtua address = 0x8
fault code = supervisor write, page not present
instruction pointer = 0x20:0xc2f62a9b
stack pointer = 0x28:0xd615ebc4
frame pointer = 0x28:0xd615ebe8
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, def32 1, gran 1
processor eflags = interrupt enabled, resume, IOPL = 0
current process = 99236 (ngctl)
[thread pid 99236 tid 100214 ]
Stopped at ng_address_path+0x4d: movl %eax,0x8(%ebx)
db> show msgbuf
.. lot of these messages
<7>TCP: [x.x.x.x]:443 to [y.y.y.y]:56747 tcpflags 0x19<FIN,PUSH,ACK>;tcp_do_segment: FIN_WAIT_1: Received 23 bytes of data after socket was closed, sending RST and removing tcpcb
db> bt
Tracing pid 99236 tid 100214 td 0xc383aaa0
ng_address_path(0,0,c332cbe2,0,c293a000,...) at ng_address_path+0x4d
ngd_connect(c334c000,c332cbe0,c383aaa0,25,d615ec60,...) at ngd_connect+0x65
soconnect(c334c000,c332cbe0,c383aaa0,c065c456,bfbfed2c,...) at soconnect+0x52
kern_connect(c383aaa0,4,c332cbe0,c332cbe0,0,...) at kern_connect+0x56
syscall(d615ed38) at syscall+0x32e
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (98, FreeBSD ELF32, connect), eip = 0x281a2783, esp = 0xbfbfecdc, ebp=0xbfbfecf8
kgdb -c vmcore.0 kernel.debug
[GDB will not be able to debug user-mode threads: /usr/lib/libthread_db.so: Undefined symbol "ps_pglobal_lookup"]
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i386-marcel-freebsd".
There is no member named pathname.
Reading symbols from /boot/kernel/acpi.ko...Reading symbols from /boot/kernel/acpi.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/acpi.ko
Reading symbols from /boot/kernel/linux.ko...Reading symbols from /boot/kernel/linux.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/linux.ko
Reading symbols from /boot/kernel/ng_socket.ko...Reading symbols from /boot/kernel/ng_socket.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/ng_socket.ko
Reading symbols from /boot/kernel/netgraph.ko...Reading symbols from /boot/kernel/netgraph.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/netgraph.ko
Reading symbols from /boot/kernel/ng_iface.ko...Reading symbols from /boot/kernel/ng_iface.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/ng_iface.ko
Reading symbols from /boot/kernel/ng_ppp.ko...Reading symbols from /boot/kernel/ng_ppp.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/ng_ppp.ko
Reading symbols from /boot/kernel/ng_bpf.ko...Reading symbols from /boot/kernel/ng_bpf.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/ng_bpf.ko
Reading symbols from /boot/kernel/ng_vjc.ko...Reading symbols from /boot/kernel/ng_vjc.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/ng_vjc.ko
Reading symbols from /boot/kernel/ng_pptpgre.ko...Reading symbols from /boot/kernel/ng_pptpgre.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/ng_pptpgre.ko
Reading symbols from /boot/kernel/ng_ksocket.ko...Reading symbols from /boot/kernel/ng_ksocket.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/ng_ksocket.ko
Reading symbols from /boot/kernel/fire_saver.ko...Reading symbols from /boot/kernel/fire_saver.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/fire_saver.ko
Reading symbols from /boot/kernel/ng_mppc.ko...Reading symbols from /boot/kernel/ng_mppc.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/ng_mppc.ko
Reading symbols from /boot/kernel/rc4.ko...Reading symbols from /boot/kernel/rc4.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/rc4.ko
Unread portion of the kernel message buffer:
Fatal trap 12: page fault while in kernel mode
cpuid = 0; apic id = 00
fault virtual address = 0x8
fault code = supervisor write, page not present
instruction pointer = 0x20:0xc2f62a9b
stack pointer = 0x28:0xd615ebc4
frame pointer = 0x28:0xd615ebe8
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, def32 1, gran 1
processor eflags = interrupt enabled, resume, IOPL = 0
current process = 99236 (ngctl)
panic: from debugger
cpuid = 0
Uptime: 4d22h59m14s
Physical memory: 501 MB
Dumping 88 MB: 73 57 41 25 9
#0 doadump () at pcpu.h:195
195 pcpu.h: No such file or directory.
in pcpu.h
(kgdb) bt
#0 doadump () at pcpu.h:195
#1 0xc06057d6 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:418
#2 0xc0605ac9 in panic (fmt=Variable "fmt" is not available.
) at /usr/src/sys/kern/kern_shutdown.c:572
#3 0xc04b2be9 in db_panic (addr=Could not find the frame base for "db_panic".
) at /usr/src/sys/ddb/db_command.c:446
#4 0xc04b3307 in db_command (last_cmdp=0xc0929614, cmd_table=0x0, dopager=1)
at /usr/src/sys/ddb/db_command.c:413
#5 0xc04b340a in db_command_loop () at /usr/src/sys/ddb/db_command.c:466
#6 0xc04b4d7e in db_trap (type=12, code=0) at /usr/src/sys/ddb/db_main.c:228
#7 0xc062e15c in kdb_trap (type=12, code=0, tf=0xd615eb84)
at /usr/src/sys/kern/subr_kdb.c:524
#8 0xc0860055 in trap_fatal (frame=0xd615eb84, eva=8)
at /usr/src/sys/i386/i386/trap.c:890
#9 0xc08602e9 in trap_pfault (frame=0xd615eb84, usermode=0, eva=8)
at /usr/src/sys/i386/i386/trap.c:812
#10 0xc0860c76 in trap (frame=0xd615eb84) at /usr/src/sys/i386/i386/trap.c:490
#11 0xc084869b in calltrap () at /usr/src/sys/i386/i386/exception.s:139
#12 0xc2f62a9b in ng_address_path (here=0x0, item=0x0,
address=0xc332cbe2 "ngctl99236:", retaddr=0)
at /usr/src/sys/modules/netgraph/netgraph/../../../netgraph/ng_base.c:3599
---Type <return> to continue, or q <return> to quit---
#13 0xc2f5cb25 in ngd_connect (so=0xc334c000, nam=0xc332cbe0, td=0xc383aaa0)
at /usr/src/sys/modules/netgraph/socket/../../../netgraph/ng_socket.c:737
#14 0xc06568f7 in soconnect (so=0xc334c000, nam=0xc332cbe0, td=0xc383aaa0)
at /usr/src/sys/kern/uipc_socket.c:765
#15 0xc065ce62 in kern_connect (td=0xc383aaa0, fd=4, sa=0xc332cbe0)
at /usr/src/sys/kern/uipc_syscalls.c:558
#16 0xc065cfe7 in connect (td=0xc383aaa0, uap=0xd615ecfc)
at /usr/src/sys/kern/uipc_syscalls.c:526
#17 0xc0860635 in syscall (frame=0xd615ed38)
at /usr/src/sys/i386/i386/trap.c:1035
#18 0xc0848700 in Xint0x80_syscall () at /usr/src/sys/i386/i386/exception.s:196
#19 0x00000033 in ?? ()
Previous frame inner to this frame (corrupt stack?)
(kgdb) f 12
#12 0xc2f62a9b in ng_address_path (here=0x0, item=0x0,
address=0xc332cbe2 "ngctl99236:", retaddr=0)
at /usr/src/sys/modules/netgraph/netgraph/../../../netgraph/ng_base.c:3599
3599 NGI_SET_NODE(item, dest);
(kgdb) list
3594 error = ng_path2noderef(here, address, &dest, &hook);
3595 if (error) {
3596 NG_FREE_ITEM(item);
3597 return (error);
3598 }
3599 NGI_SET_NODE(item, dest);
3600 if ( hook) {
3601 NG_HOOK_REF(hook); /* don't let it go while on the queue */
3602 NGI_SET_HOOK(item, hook);
3603 }
Copyright (c) 1992-2008 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 7.0-STABLE #6: Sat May 10 05:27:18 PDT 2008
root at xxx:/usr/obj/usr/src/sys/XXX
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: Intel(R) Pentium(R) 4 CPU 2.40GHz (2394.01-MHz 686-class CPU)
Origin = "GenuineIntel" Id = 0xf25 Stepping = 5
Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
Features2=0x4400<CNXT-ID,xTPR>
Logical CPUs per core: 2
real memory = 534970368 (510 MB)
avail memory = 513785856 (489 MB)
ACPI APIC Table: <INTEL D875BZLK>
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
cpu0 (BSP): APIC ID: 0
cpu1 (AP): APIC ID: 1
[chifung at marx /var/log]$ dmesg | less
Copyright (c) 1992-2008 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 7.0-STABLE #6: Sat May 10 05:27:18 PDT 2008
root at xxx:/usr/obj/usr/src/sys/XXX
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: Intel(R) Pentium(R) 4 CPU 2.40GHz (2394.01-MHz 686-class CPU)
Origin = "GenuineIntel" Id = 0xf25 Stepping = 5
Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
Features2=0x4400<CNXT-ID,xTPR>
Logical CPUs per core: 2
real memory = 534970368 (510 MB)
avail memory = 513785856 (489 MB)
ACPI APIC Table: <INTEL D875BZLK>
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
cpu0 (BSP): APIC ID: 0
cpu1 (AP): APIC ID: 1
ioapic0 <Version 2.0> irqs 0-23 on motherboard
kbd1 at kbdmux0
acpi0: <INTEL D875BZLK> on motherboard
acpi0: [ITHREAD]
acpi0: Power Button (fixed)
acpi0: reservation of 0, a0000 (3) failed
acpi0: reservation of 100000, 1ff00000 (3) failed
Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0
cpu0: <ACPI CPU> on acpi0
p4tcc0: <CPU Frequency Thermal Control> on cpu0
cpu1: <ACPI CPU> on acpi0
p4tcc1: <CPU Frequency Thermal Control> on cpu1
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
agp0: <Intel 82875P host to AGP bridge> on hostb0
pcib1: <ACPI PCI-PCI bridge> at device 1.0 on pci0
pci1: <ACPI PCI bus> on pcib1
vgapci0: <VGA-compatible display> mem 0xf5000000-0xf5ffffff,0xf8000000-0xfbffffff irq 16 at device 0.0 on pci1
pcib2: <ACPI PCI-PCI bridge> at device 3.0 on pci0
pci2: <ACPI PCI bus> on pcib2
em0: <Intel(R) PRO/1000 Network Connection 6.9.0> port 0xac00-0xac1f mem 0xf7000000-0xf701ffff irq 18 at device 1.0 on pci2
em0: [FILTER]
em0: Ethernet address: 00:07:e9:3b:bc:37
uhci0: <Intel 82801EB (ICH5) USB controller USB-A> port 0xcc00-0xcc1f irq 16 at device 29.0 on pci0
uhci0: [GIANT-LOCKED]
uhci0: [ITHREAD]
usb0: <Intel 82801EB (ICH5) USB controller USB-A> on uhci0
usb0: USB revision 1.0
uhub0: <Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb0
uhub0: 2 ports with 2 removable, self powered
uhci1: <Intel 82801EB (ICH5) USB controller USB-B> port 0xd000-0xd01f irq 19 at device 29.1 on pci0
uhci1: [GIANT-LOCKED]
uhci1: [ITHREAD]
usb1: <Intel 82801EB (ICH5) USB controller USB-B> on uhci1
usb1: USB revision 1.0
uhub1: <Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb1
uhub1: 2 ports with 2 removable, self powered
uhci2: <Intel 82801EB (ICH5) USB controller USB-C> port 0xd400-0xd41f irq 18 at device 29.2 on pci0
uhci2: [GIANT-LOCKED]
uhci2: [ITHREAD]
usb2: <Intel 82801EB (ICH5) USB controller USB-C> on uhci2
usb2: USB revision 1.0
uhub2: <Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb2
uhub2: 2 ports with 2 removable, self powered
uhci3: <Intel 82801EB (ICH5) USB controller USB-D> port 0xd800-0xd81f irq 16 at device 29.3 on pci0
uhci3: [GIANT-LOCKED]
uhci3: [ITHREAD]
usb3: <Intel 82801EB (ICH5) USB controller USB-D> on uhci3
usb3: USB revision 1.0
uhub3: <Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb3
uhub3: 2 ports with 2 removable, self powered
ehci0: <Intel 82801EB/R (ICH5) USB 2.0 controller> mem 0xf7200000-0xf72003ff irq 23 at device 29.7 on pci0
ehci0: [GIANT-LOCKED]
ehci0: [ITHREAD]
usb4: EHCI version 1.0
usb4: companion controllers, 2 ports each: usb0 usb1 usb2 usb3
usb4: <Intel 82801EB/R (ICH5) USB 2.0 controller> on ehci0
usb4: USB revision 2.0
uhub4: <Intel EHCI root hub, class 9/0, rev 2.00/1.00, addr 1> on usb4
uhub4: 8 ports with 8 removable, self powered
pcib3: <ACPI PCI-PCI bridge> at device 30.0 on pci0
pci3: <ACPI PCI bus> on pcib3
rl0: <RealTek 8139 10/100BaseTX> port 0xbc00-0xbc7f mem 0xf7100000-0xf710007f irq 22 at device 1.0 on pci3
miibus0: <MII bus> on rl0
rlphy0: <RealTek internal media interface> PHY 0 on miibus0
rlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
rl0: Ethernet address: 00:a0:4b:04:91:b4
rl0: [ITHREAD]
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <Intel ICH5 UDMA100 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xffa0-0xffaf at device 31.1 on pci0
ata0: <ATA channel 0> on atapci0
ata0: [ITHREAD]
ata1: <ATA channel 1> on atapci0
ata1: [ITHREAD]
atapci1: <Intel ICH5 SATA150 controller> port 0xec00-0xec07,0xe800-0xe803,0xe400-0xe407,0xe000-0xe003,0xdc00-0xdc0f irq 18 at device 31.2 on pci0
atapci1: [ITHREAD]
ata2: <ATA channel 0> on atapci1
ata2: [ITHREAD]
ata3: <ATA channel 1> on atapci1
ata3: [ITHREAD]
pci0: <serial bus, SMBus> at device 31.3 (no driver attached)
acpi_button0: <Sleep Button> on acpi0
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
atkbd0: [ITHREAD]
psm0: <PS/2 Mouse> irq 12 on atkbdc0
psm0: [GIANT-LOCKED]
psm0: [ITHREAD]
psm0: model Generic PS/2 mouse, device ID 0
fdc0: <floppy drive controller> port 0x3f0-0x3f1,0x3f2-0x3f3,0x3f4-0x3f5,0x3f7 irq 6 drq 2 on acpi0
fdc0: [FILTER]
fd0: <1440-KB 3.5" drive> on fdc0 drive 0
sio0: configured irq 4 not in bitmap of probed irqs 0
sio0: port may not be enabled
sio0: configured irq 4 not in bitmap of probed irqs 0
sio0: port may not be enabled
sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
sio0: type 16550A
sio0: [FILTER]
cryptosoft0: <software crypto> on motherboard
pmtimer0 on isa0
ppc0: <Parallel port> at port 0x378-0x37f irq 7 on isa0
ppc0: Generic chipset (EPP/NIBBLE) in COMPATIBLE mode
ppbus0: <Parallel port bus> on ppc0
ppbus0: [ITHREAD]
plip0: <PLIP network interface> on ppbus0
lpt0: <Printer> on ppbus0
lpt0: Interrupt-driven port
ppi0: <Parallel I/O> on ppbus0
ppc0: [GIANT-LOCKED]
ppc0: [ITHREAD]
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
sio1: configured irq 3 not in bitmap of probed irqs 0
sio1: port may not be enabled
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
Timecounters tick every 1.000 msec
IPsec: Initialized Security Association Processing.
ad0: 57241MB <WDC WD600LB-55DNA0 77.07W77> at ata0-master UDMA100
SMP: AP CPU #1 Launched!
Trying to mount root from ufs:/dev/ad0s1a
WARNING: /var/backups was not properly dismounted
WARNING: attempt to net_add_domain(netgraph) after domainfinalize()
WARNING: /crypt was not properly dismounted
>How-To-Repeat:
Setup VPN using mpd between 2 servers.
>Fix:
>Release-Note:
>Audit-Trail:
>Unformatted:
More information about the freebsd-bugs
mailing list