kern/143349: kernel panic: vm_page_free: freeing busy page

[Erik Klavon]

>Number:         143349
>Category:       kern
>Synopsis:       kernel panic: vm_page_free: freeing busy page
>Confidential:   no
>Severity:       serious
>Priority:       low
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Fri Jan 29 19:10:09 UTC 2010
>Closed-Date:
>Last-Modified:
>Originator:     Erik Klavon
>Release:        8.0p1
>Organization:
University of California, Berkeley
>Environment:
FreeBSD doecev1cp.net.berkeley.edu 8.0-RELEASE-p1 FreeBSD 8.0-RELEASE-p1 #4: Mon Jan  4 15:25:35 PST 2010     root at master1cp.net.berkeley.edu:/usr/obj/usr/src/sys/WIRELESS  amd64

>Description:
A kernel panic occurred today on a machine with an uptime of about 5 days.
Console output including a stack trace is below. I was not able to obtain a
core dump after this panic. The machine froze when I attempted to generate
a code dump and had to be power cycled. The machine acts as a NAT device and
was carrying user traffic at the time of the panic.

I have conducted load tests of an identical machine in a test environment and
I have not been able to reproduce this kernel panic. We've been running 8.0p1
in this set up on multiple machines for a couple of months. This is the first
kernel panic of this type we've seen.

Jan 28 15:08:58 doecev1cp dhcpd: send_packet: Host is down
Jan 28 15:09:08 doecev1cp last message repeated 4 times
Jan 28 15:25:48 doecev1cp dhcpd: send_packet: Host is down
Jan 28 15:58:45 doecev1cp dhcpd: send_packet: Host is down
Jan 28 16:39:30 doecev1cp dhcpd: send_packet: Host is down
Jan 28 18:59:33 doecev1cp kernel: arp: unknown hardware address format (0x0000)
Jan 28 18:59:33 doecev1cp kernel: arp: unknown hardware address format (0x0000)
vm_page_free: pindex(246), busy(96), VPO_BUSY(0), hold(0)
panic: vm_page_free: freeing busy page
cpuid = 1
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2a
panic() at panic+0x182
vm_page_free_toq() at vm_page_free_toq+0x1f3
vm_object_terminate() at vm_object_terminate+0x6b
vm_object_deallocate() at vm_object_deallocate+0x44c
_vm_map_unlock() at _vm_map_unlock+0x70
vm_map_remove() at vm_map_remove+0x5f
vmspace_exit() at vmspace_exit+0xc4
exit1() at exit1+0x361
sys_exit() at sys_exit+0xe
syscall() at syscall+0x246
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (1, FreeBSD ELF64, sys_exit), rip = 0x80111e21c, rsp = 0x7fffffffe888, rbp = 0x7fffffffe91c ---
KDB: enter: panic
[thread pid 20050 tid 103053 ]
Stopped at      kdb_enter+0x3d: movq    $0,0x6bdb80(%rip)

kernel configuration follows.

include GENERIC 
ident WIRELESS 

options KDB
options DDB
options KDB_TRACE
options BREAK_TO_DEBUGGER

options IPSEC
options IPSEC_FILTERTUNNEL
device crypto

options DEVICE_POLLING

options IPFIREWALL
options IPFIREWALL_VERBOSE
options IPFIREWALL_VERBOSE_LIMIT=100
options IPFIREWALL_FORWARD
options DUMMYNET

options IPDIVERT
options HZ=2500

device carp

dmesg output follows.

Copyright (c) 1992-2009 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 8.0-RELEASE-p1 #4: Mon Jan  4 15:25:35 PST 2010
    root at master1cp.net.berkeley.edu:/usr/obj/usr/src/sys/WIRELESS
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: AMD Opteron(tm) Processor 252 (2589.84-MHz K8-class CPU)
  Origin = "AuthenticAMD"  Id = 0x20f51  Stepping = 1
  Features=0x78bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2>
  Features2=0x1<SSE3>
  AMD Features=0xe2500800<SYSCALL,NX,MMX+,FFXSR,LM,3DNow!+,3DNow!>
  AMD Features2=0x1<LAHF>
real memory  = 4294967296 (4096 MB)
avail memory = 4065366016 (3877 MB)
ACPI APIC Table: <A M I  OEMAPIC >
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
FreeBSD/SMP: 2 package(s) x 1 core(s)
 cpu0 (BSP): APIC ID:  0
 cpu1 (AP): APIC ID:  1
MADT: Forcing active-low polarity and level trigger for SCI
ioapic0 <Version 1.1> irqs 0-23 on motherboard
ioapic1 <Version 1.1> irqs 24-27 on motherboard
ioapic2 <Version 1.1> irqs 28-31 on motherboard
kbd1 at kbdmux0
cryptosoft0: <software crypto> on motherboard
acpi0: <A M I OEMRSDT> on motherboard
acpi0: [ITHREAD]
acpi0: Power Button (fixed)
acpi0: reservation of 0, a0000 (3) failed
acpi0: reservation of 100000, fbf00000 (3) failed
Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> at device 6.0 on pci0
pci3: <ACPI PCI bus> on pcib1
ohci0: <OHCI (generic) USB controller> mem 0xfeafc000-0xfeafcfff irq 19 at device 0.0 on pci3
ohci0: [ITHREAD]
usbus0: <OHCI (generic) USB controller> on ohci0
ohci1: <OHCI (generic) USB controller> mem 0xfeafd000-0xfeafdfff irq 19 at device 0.1 on pci3
ohci1: [ITHREAD]
usbus1: <OHCI (generic) USB controller> on ohci1
atapci0: <SiI 3114 SATA150 controller> port 0xbc00-0xbc07,0xb880-0xb883,0xb800-0xb807,0xac00-0xac03,0xa880-0xa88f mem 0xfeafec00-0xfeafefff irq 19 at device 5.0 on pci3
atapci0: [ITHREAD]
ata2: <ATA channel 0> on atapci0
ata2: [ITHREAD]
ata3: <ATA channel 1> on atapci0
ata3: [ITHREAD]
ata4: <ATA channel 2> on atapci0
ata4: [ITHREAD]
ata5: <ATA channel 3> on atapci0
ata5: [ITHREAD]
vgapci0: <VGA-compatible display> port 0xb000-0xb0ff mem 0xfd000000-0xfdffffff,0xfeaff000-0xfeafffff irq 18 at device 6.0 on pci3
fxp0: <Intel 82551 Pro/100 Ethernet> port 0xa800-0xa83f mem 0xfeafb000-0xfeafbfff,0xfeaa0000-0xfeabffff irq 18 at device 8.0 on pci3
miibus0: <MII bus> on fxp0
inphy0: <i82555 10/100 media interface> PHY 1 on miibus0
inphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
fxp0: Ethernet address: 00:e0:81:32:00:b2
fxp0: [ITHREAD]
isab0: <PCI-ISA bridge> at device 7.0 on pci0
isa0: <ISA bus> on isab0
atapci1: <AMD 8111 UDMA133 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xffa0-0xffaf at device 7.1 on pci0
ata0: <ATA channel 0> on atapci1
ata0: [ITHREAD]
ata1: <ATA channel 1> on atapci1
ata1: [ITHREAD]
pci0: <serial bus, SMBus> at device 7.2 (no driver attached)
pci0: <bridge> at device 7.3 (no driver attached)
pcib2: <ACPI PCI-PCI bridge> at device 10.0 on pci0
pci2: <ACPI PCI bus> on pcib2
pci2: <mass storage, SCSI> at device 6.0 (no driver attached)
pci2: <mass storage, SCSI> at device 6.1 (no driver attached)
bge0: <Broadcom NetXtreme Gigabit Ethernet Controller, ASIC rev. 0x2003> mem 0xfc8b0000-0xfc8bffff,0xfc8a0000-0xfc8affff irq 24 at device 9.0 on pci2
miibus1: <MII bus> on bge0
brgphy0: <BCM5704 10/100/1000baseTX PHY> PHY 1 on miibus1
brgphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto
bge0: Ethernet address: 00:e0:81:32:00:24
bge0: [ITHREAD]
bge1: <Broadcom NetXtreme Gigabit Ethernet Controller, ASIC rev. 0x2003> mem 0xfc8e0000-0xfc8effff,0xfc8d0000-0xfc8dffff irq 25 at device 9.1 on pci2
miibus2: <MII bus> on bge1
brgphy1: <BCM5704 10/100/1000baseTX PHY> PHY 1 on miibus2
brgphy1:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto
bge1: Ethernet address: 00:e0:81:32:00:25
bge1: [ITHREAD]
pcib3: <ACPI PCI-PCI bridge> at device 11.0 on pci0
pci1: <ACPI PCI bus> on pcib3
acpi_button0: <Power Button> on acpi0
atrtc0: <AT realtime clock> port 0x70-0x71 on acpi0
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
uart0: [FILTER]
uart0: console (9600,n,8,1)
uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0
uart1: [FILTER]
ACPI Warning: \\_SB_.PCI0.SBRG.FDC_._FDE: Return type mismatch - found Package, expected Buffer 20090521 nspredef-1051
fdc0: <floppy drive controller (FDE)> port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0
fdc0: [FILTER]
fd0: <1440-KB 3.5" drive> on fdc0 drive 0
acpi_hpet0: <High Precision Event Timer> iomem 0xfec01000-0xfec013ff irq 0,8 on acpi0
Timecounter "HPET" frequency 14318180 Hz quality 900
cpu0: <ACPI CPU> on acpi0
acpi_throttle0: <ACPI CPU Throttling> on cpu0
powernow0: <Cool`n'Quiet K8> on cpu0
cpu1: <ACPI CPU> on acpi0
powernow1: <Cool`n'Quiet K8> on cpu1
orm0: <ISA Option ROMs> at iomem 0xc0000-0xc7fff,0xc8000-0xcc7ff,0xcc800-0xcdfff,0xce000-0xcf7ff,0xcf800-0xd07ff on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
atkbd0: [ITHREAD]
ppc0: cannot reserve I/O port range
Timecounters tick every 1.200 msec
IPsec: Initialized Security Association Processing.
ipfw2 (+ipv6) initialized, divert enabled, nat loadable, rule-based forwarding enabled, default to deny, logging disabled
usbus0: 12Mbps Full Speed USB v1.0
usbus1: 12Mbps Full Speed USB v1.0
acd0: CDROM <SONY CD-ROM CDU5215/7YS1> at ata0-master UDMA33
ad4: 76319MB <WDC WD800AAJS-22L7A0 01.03E01> at ata2-master SATA150
ugen0.1: <AMD> at usbus0
uhub0: <AMD OHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
ugen1.1: <AMD> at usbus1
uhub1: <AMD OHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus1
ad6: 76319MB <WDC WD800AAJS-22L7A0 01.03E01> at ata3-master SATA150
SMP: AP CPU #1 Launched!
uhub0: 3 ports with 3 removable, self powered
uhub1: 3 ports with 3 removable, self powered
GEOM_MIRROR: Device mirror/gm0 launched (1/2).
GEOM_MIRROR: Device gm0: rebuilding provider ad4.
GEOM: mirror/gm0s1: geometry does not match label (16h,63s != 255h,63s).
Trying to mount root from ufs:/dev/mirror/gm0s1a
WARNING: / was not properly dismounted
WARNING: /tmp was not properly dismounted
WARNING: /usr was not properly dismounted
WARNING: /var was not properly dismounted
/var: mount pending error: blocks 36 files 0
bridge1950: Ethernet address: 06:58:81:d4:6d:19
bridge1951: Ethernet address: be:4f:78:ea:18:a6
bridge1952: Ethernet address: 86:c7:de:33:f6:ee
bge1: 2 link states coalesced
WARNING: attempt to domain_add(netgraph) after domainfinalize()
>How-To-Repeat:
Unknown.
>Fix:


>Release-Note:
>Audit-Trail:
>Unformatted: