ZFS Panic after freebsd-update

[Scott Sipe]

*** Sorry for partial first message! (gmail sent after multiple returns
apparently?) ***

Hello,

I have not had much time to research this problem yet, so please let me
know what further information I might be able to provide.

This weekend I attempted to upgrade a computer from 8.2-RELEASE-p3 to 8.4
using freebsd-update. After I rebooted to test the new kernel, I got a
panic. I had to take a picture of the screen. Here's a condensed version:

panic: page fault
cpuid = 1
KDB: stack backtrace:
#0 kdb_backtrace
#1 panic
#2 trap_fatal
#3 trap_pfault
#4 trap
#5 calltrap
#6 vdev_mirror_child_select
#7 ved_mirror_io_start
#8 zio_vdev_io_start
#9 zio_execute
#10 arc_read
#11 dbuf_read
#12 dbuf_findbp
#13 dbuf_hold_impl
#14 dbuf_hold
#15 dnode_hold_impl
#16 dnu_buf_hold
#17 zap_lockdir
Uptime: 5s
Cannot dump. Device not defined or unavailable.
Automatic reboot in 15 seconds - press a key on the console to abort

uname -a from before (and after) the reboot:

FreeBSD xeon 8.2-RELEASE-p3 FreeBSD 8.2-RELEASE-p3 #0: Tue Sep 27 18:45:57
UTC 2011     root at amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC
 amd64

dmesg is attached.

I was able to reboot to the old kernel and am up and running back on 8.2
right now.

Any thoughts?

Thanks,
Scott
-------------- next part --------------
Copyright (c) 1992-2011 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
	The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 8.2-RELEASE-p3 #0: Tue Sep 27 18:45:57 UTC 2011
    root at amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz (2266.76-MHz K8-class CPU)
  Origin = "GenuineIntel"  Id = 0x106a5  Family = 6  Model = 1a  Stepping = 5
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0x9ce3bd<SSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT>
  AMD Features=0x28100800<SYSCALL,NX,RDTSCP,LM>
  AMD Features2=0x1<LAHF>
  TSC: P-state invariant
real memory  = 18253611008 (17408 MB)
avail memory = 16513347584 (15748 MB)
ACPI APIC Table: <031710 APIC1617>
FreeBSD/SMP: Multiprocessor System Detected: 16 CPUs
FreeBSD/SMP: 2 package(s) x 4 core(s) x 2 SMT threads
 cpu0 (BSP): APIC ID:  0
 cpu1 (AP): APIC ID:  1
 cpu2 (AP): APIC ID:  2
 cpu3 (AP): APIC ID:  3
 cpu4 (AP): APIC ID:  4
 cpu5 (AP): APIC ID:  5
 cpu6 (AP): APIC ID:  6
 cpu7 (AP): APIC ID:  7
 cpu8 (AP): APIC ID: 16
 cpu9 (AP): APIC ID: 17
 cpu10 (AP): APIC ID: 18
 cpu11 (AP): APIC ID: 19
 cpu12 (AP): APIC ID: 20
 cpu13 (AP): APIC ID: 21
 cpu14 (AP): APIC ID: 22
 cpu15 (AP): APIC ID: 23
ioapic0 <Version 2.0> irqs 0-23 on motherboard
ioapic1 <Version 2.0> irqs 24-47 on motherboard
kbd1 at kbdmux0
acpi0: <031710 XSDT1617> on motherboard
acpi0: [ITHREAD]
acpi0: Power Button (fixed)
acpi0: reservation of 0, a0000 (3) failed
acpi0: reservation of 100000, bff00000 (3) failed
Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0
cpu0: <ACPI CPU> on acpi0
ACPI Warning: Incorrect checksum in table [OEMB] - 0xAD, should be 0xAA (20101013/tbutils-354)
cpu1: <ACPI CPU> on acpi0
cpu2: <ACPI CPU> on acpi0
cpu3: <ACPI CPU> on acpi0
cpu4: <ACPI CPU> on acpi0
cpu5: <ACPI CPU> on acpi0
cpu6: <ACPI CPU> on acpi0
cpu7: <ACPI CPU> on acpi0
cpu8: <ACPI CPU> on acpi0
cpu9: <ACPI CPU> on acpi0
cpu10: <ACPI CPU> on acpi0
cpu11: <ACPI CPU> on acpi0
cpu12: <ACPI CPU> on acpi0
cpu13: <ACPI CPU> on acpi0
cpu14: <ACPI CPU> on acpi0
cpu15: <ACPI CPU> on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> at device 1.0 on pci0
pci10: <ACPI PCI bus> on pcib1
pcib2: <ACPI PCI-PCI bridge> at device 3.0 on pci0
pci9: <ACPI PCI bus> on pcib2
pcib3: <ACPI PCI-PCI bridge> at device 7.0 on pci0
pci8: <ACPI PCI bus> on pcib3
pcib4: <PCI-PCI bridge> at device 8.0 on pci0
pci7: <PCI bus> on pcib4
pcib5: <PCI-PCI bridge> at device 9.0 on pci0
pci6: <PCI bus> on pcib5
pcib6: <PCI-PCI bridge> at device 10.0 on pci0
pci5: <PCI bus> on pcib6
pci0: <base peripheral, interrupt controller> at device 20.0 (no driver attached)
pci0: <base peripheral, interrupt controller> at device 20.1 (no driver attached)
pci0: <base peripheral, interrupt controller> at device 20.2 (no driver attached)
pci0: <base peripheral, interrupt controller> at device 20.3 (no driver attached)
pci0: <base peripheral> at device 22.0 (no driver attached)
pci0: <base peripheral> at device 22.1 (no driver attached)
pci0: <base peripheral> at device 22.2 (no driver attached)
pci0: <base peripheral> at device 22.3 (no driver attached)
pci0: <base peripheral> at device 22.4 (no driver attached)
pci0: <base peripheral> at device 22.5 (no driver attached)
pci0: <base peripheral> at device 22.6 (no driver attached)
pci0: <base peripheral> at device 22.7 (no driver attached)
uhci0: <Intel 82801JI (ICH10) USB controller USB-D> port 0xa400-0xa41f irq 16 at device 26.0 on pci0
uhci0: [ITHREAD]
uhci0: LegSup = 0x2f00
usbus0: <Intel 82801JI (ICH10) USB controller USB-D> on uhci0
uhci1: <Intel 82801JI (ICH10) USB controller USB-E> port 0xa480-0xa49f irq 21 at device 26.1 on pci0
uhci1: [ITHREAD]
uhci1: LegSup = 0x2f00
usbus1: <Intel 82801JI (ICH10) USB controller USB-E> on uhci1
uhci2: <Intel 82801JI (ICH10) USB controller USB-F> port 0xa800-0xa81f irq 19 at device 26.2 on pci0
uhci2: [ITHREAD]
uhci2: LegSup = 0x2f00
usbus2: <Intel 82801JI (ICH10) USB controller USB-F> on uhci2
ehci0: <Intel 82801JI (ICH10) USB 2.0 controller USB-B> mem 0xfbcf4000-0xfbcf43ff irq 18 at device 26.7 on pci0
ehci0: [ITHREAD]
usbus3: EHCI version 1.0
usbus3: <Intel 82801JI (ICH10) USB 2.0 controller USB-B> on ehci0
pcib7: <ACPI PCI-PCI bridge> irq 17 at device 28.0 on pci0
pci4: <ACPI PCI bus> on pcib7
pcib8: <ACPI PCI-PCI bridge> irq 17 at device 28.4 on pci0
pci3: <ACPI PCI bus> on pcib8
em0: <Intel(R) PRO/1000 Network Connection 7.1.9> port 0xec00-0xec1f mem 0xfbee0000-0xfbefffff,0xfbedc000-0xfbedffff irq 16 at device 0.0 on pci3
em0: Using MSIX interrupts with 3 vectors
em0: [ITHREAD]
em0: [ITHREAD]
em0: [ITHREAD]
em0: Ethernet address: ***
pcib9: <ACPI PCI-PCI bridge> irq 16 at device 28.5 on pci0
pci2: <ACPI PCI bus> on pcib9
em1: <Intel(R) PRO/1000 Network Connection 7.1.9> port 0xdc00-0xdc1f mem 0xfbde0000-0xfbdfffff,0xfbddc000-0xfbddffff irq 17 at device 0.0 on pci2
em1: Using MSIX interrupts with 3 vectors
em1: [ITHREAD]
em1: [ITHREAD]
em1: [ITHREAD]
em1: Ethernet address: ***
uhci3: <Intel 82801JI (ICH10) USB controller USB-A> port 0xa880-0xa89f irq 23 at device 29.0 on pci0
uhci3: [ITHREAD]
uhci3: LegSup = 0x2f00
usbus4: <Intel 82801JI (ICH10) USB controller USB-A> on uhci3
uhci4: <Intel 82801JI (ICH10) USB controller USB-B> port 0xac00-0xac1f irq 19 at device 29.1 on pci0
uhci4: [ITHREAD]
uhci4: LegSup = 0x2f00
usbus5: <Intel 82801JI (ICH10) USB controller USB-B> on uhci4
uhci5: <Intel 82801JI (ICH10) USB controller USB-C> port 0xb000-0xb01f irq 18 at device 29.2 on pci0
uhci5: [ITHREAD]
uhci5: LegSup = 0x2f00
usbus6: <Intel 82801JI (ICH10) USB controller USB-C> on uhci5
ehci1: <Intel 82801JI (ICH10) USB 2.0 controller USB-A> mem 0xfbcf6000-0xfbcf63ff irq 23 at device 29.7 on pci0
ehci1: [ITHREAD]
usbus7: EHCI version 1.0
usbus7: <Intel 82801JI (ICH10) USB 2.0 controller USB-A> on ehci1
pcib10: <ACPI PCI-PCI bridge> at device 30.0 on pci0
pci1: <ACPI PCI bus> on pcib10
vgapci0: <VGA-compatible display> port 0xcc00-0xcc7f mem 0xfb000000-0xfb7fffff,0xfafe0000-0xfaffffff irq 16 at device 0.0 on pci1
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
ahci0: <Intel ICH10 AHCI SATA controller> port 0xb880-0xb887,0xb800-0xb803,0xb480-0xb487,0xb400-0xb403,0xb080-0xb09f mem 0xfbcf8000-0xfbcf87ff irq 19 at device 31.2 on pci0
ahci0: [ITHREAD]
ahci0: AHCI v1.20 with 6 3Gbps ports, Port Multiplier not supported
ahcich0: <AHCI channel> at channel 0 on ahci0
ahcich0: [ITHREAD]
ahcich1: <AHCI channel> at channel 1 on ahci0
ahcich1: [ITHREAD]
ahcich2: <AHCI channel> at channel 2 on ahci0
ahcich2: [ITHREAD]
ahcich3: <AHCI channel> at channel 3 on ahci0
ahcich3: [ITHREAD]
ahcich4: <AHCI channel> at channel 4 on ahci0
ahcich4: [ITHREAD]
ahcich5: <AHCI channel> at channel 5 on ahci0
ahcich5: [ITHREAD]
pci0: <serial bus, SMBus> at device 31.3 (no driver attached)
acpi_button0: <Power Button> on acpi0
atrtc0: <AT realtime clock> port 0x70-0x71 irq 8 on acpi0
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
uart0: [FILTER]
uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0
uart1: [FILTER]
acpi_hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 14318180 Hz quality 900
qpi0: <QPI system bus> on motherboard
pcib11: <QPI Host-PCI bridge> pcibus 255 on qpi0
pci255: <PCI bus> on pcib11
pcib12: <QPI Host-PCI bridge> pcibus 254 on qpi0
pci254: <PCI bus> on pcib12
orm0: <ISA Option ROM> at iomem 0xc0000-0xc7fff on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: CGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3d0-0x3db iomem 0xb8000-0xbffff on isa0
atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
atkbd0: [ITHREAD]
ppc0: cannot reserve I/O port range
coretemp0: <CPU On-Die Thermal Sensors> on cpu0
est0: <Enhanced SpeedStep Frequency Control> on cpu0
p4tcc0: <CPU Frequency Thermal Control> on cpu0
coretemp1: <CPU On-Die Thermal Sensors> on cpu1
est1: <Enhanced SpeedStep Frequency Control> on cpu1
p4tcc1: <CPU Frequency Thermal Control> on cpu1
coretemp2: <CPU On-Die Thermal Sensors> on cpu2
est2: <Enhanced SpeedStep Frequency Control> on cpu2
p4tcc2: <CPU Frequency Thermal Control> on cpu2
coretemp3: <CPU On-Die Thermal Sensors> on cpu3
est3: <Enhanced SpeedStep Frequency Control> on cpu3
p4tcc3: <CPU Frequency Thermal Control> on cpu3
coretemp4: <CPU On-Die Thermal Sensors> on cpu4
est4: <Enhanced SpeedStep Frequency Control> on cpu4
p4tcc4: <CPU Frequency Thermal Control> on cpu4
coretemp5: <CPU On-Die Thermal Sensors> on cpu5
est5: <Enhanced SpeedStep Frequency Control> on cpu5
p4tcc5: <CPU Frequency Thermal Control> on cpu5
coretemp6: <CPU On-Die Thermal Sensors> on cpu6
est6: <Enhanced SpeedStep Frequency Control> on cpu6
p4tcc6: <CPU Frequency Thermal Control> on cpu6
coretemp7: <CPU On-Die Thermal Sensors> on cpu7
est7: <Enhanced SpeedStep Frequency Control> on cpu7
p4tcc7: <CPU Frequency Thermal Control> on cpu7
coretemp8: <CPU On-Die Thermal Sensors> on cpu8
est8: <Enhanced SpeedStep Frequency Control> on cpu8
p4tcc8: <CPU Frequency Thermal Control> on cpu8
coretemp9: <CPU On-Die Thermal Sensors> on cpu9
est9: <Enhanced SpeedStep Frequency Control> on cpu9
p4tcc9: <CPU Frequency Thermal Control> on cpu9
coretemp10: <CPU On-Die Thermal Sensors> on cpu10
est10: <Enhanced SpeedStep Frequency Control> on cpu10
p4tcc10: <CPU Frequency Thermal Control> on cpu10
coretemp11: <CPU On-Die Thermal Sensors> on cpu11
est11: <Enhanced SpeedStep Frequency Control> on cpu11
p4tcc11: <CPU Frequency Thermal Control> on cpu11
coretemp12: <CPU On-Die Thermal Sensors> on cpu12
est12: <Enhanced SpeedStep Frequency Control> on cpu12
p4tcc12: <CPU Frequency Thermal Control> on cpu12
coretemp13: <CPU On-Die Thermal Sensors> on cpu13
est13: <Enhanced SpeedStep Frequency Control> on cpu13
p4tcc13: <CPU Frequency Thermal Control> on cpu13
coretemp14: <CPU On-Die Thermal Sensors> on cpu14
est14: <Enhanced SpeedStep Frequency Control> on cpu14
p4tcc14: <CPU Frequency Thermal Control> on cpu14
coretemp15: <CPU On-Die Thermal Sensors> on cpu15
est15: <Enhanced SpeedStep Frequency Control> on cpu15
p4tcc15: <CPU Frequency Thermal Control> on cpu15
ZFS filesystem version 4
ZFS storage pool version 15
Timecounters tick every 1.000 msec
vboxdrv: fAsync=0 offMin=0x734 offMax=0x8c8
usbus0: 12Mbps Full Speed USB v1.0
usbus1: 12Mbps Full Speed USB v1.0
usbus2: 12Mbps Full Speed USB v1.0
usbus3: 480Mbps High Speed USB v2.0
usbus4: 12Mbps Full Speed USB v1.0
usbus5: 12Mbps Full Speed USB v1.0
usbus6: 12Mbps Full Speed USB v1.0
usbus7: 480Mbps High Speed USB v2.0
ugen0.1: <Intel> at usbus0
uhub0: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
ugen1.1: <Intel> at usbus1
uhub1: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus1
ugen2.1: <Intel> at usbus2
uhub2: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus2
ugen3.1: <Intel> at usbus3
uhub3: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus3
ugen4.1: <Intel> at usbus4
uhub4: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus4
ugen5.1: <Intel> at usbus5
uhub5: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus5
ugen6.1: <Intel> at usbus6
uhub6: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus6
ugen7.1: <Intel> at usbus7
uhub7: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus7
ada0 at ahcich0 bus 0 scbus0 target 0 lun 0
ada0: <WDC WD2003FYYS-02W0B0 01.01D01> ATA-8 SATA 2.x device
ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada0: Command Queueing enabled
ada0: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada1 at ahcich1 bus 0 scbus1 target 0 lun 0
ada1: <WDC WD2003FYYS-02W0B0 01.01D01> ATA-8 SATA 2.x device
ada1: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada1: Command Queueing enabled
ada1: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada2 at ahcich2 bus 0 scbus2 target 0 lun 0
ada2: <WDC WD2002FYPS-02W3B1 04.01G02> ATA-8 SATA 2.x device
ada2: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada2: Command Queueing enabled
ada2: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada3 at ahcich3 bus 0 scbus3 target 0 lun 0
ada3: <WDC WD2002FYPS-02W3B1 04.01G02> ATA-8 SATA 2.x device
ada3: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada3: Command Queueing enabled
ada3: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
SMP: AP CPU #1 Launched!
SMP: AP CPU #14 Launched!
SMP: AP CPU #5 Launched!
SMP: AP CPU #13 Launched!
SMP: AP CPU #2 Launched!
SMP: AP CPU #12 Launched!
SMP: AP CPU #4 Launched!
SMP: AP CPU #8 Launched!
SMP: AP CPU #6 Launched!
SMP: AP CPU #11 Launched!
SMP: AP CPU #3 Launched!
SMP: AP CPU #15 Launched!
SMP: AP CPU #7 Launched!
SMP: AP CPU #10 Launched!
SMP: AP CPU #9 Launched!
uhub0: 2 ports with 2 removable, self powered
uhub1: 2 ports with 2 removable, self powered
uhub2: 2 ports with 2 removable, self powered
uhub4: 2 ports with 2 removable, self powered
uhub5: 2 ports with 2 removable, self powered
uhub6: 2 ports with 2 removable, self powered
Root mount waiting for: usbus7 usbus3
Root mount waiting for: usbus7 usbus3
uhub3: 6 ports with 6 removable, self powered
uhub7: 6 ports with 6 removable, self powered
Root mount waiting for: usbus7
ugen7.2: <American Megatrends Inc.> at usbus7
Trying to mount root from zfs:zroot
ugen4.2: <Dell> at usbus4
ukbd0: <EP1 Interrupt> on usbus4
kbd2 at ukbd0
ugen0.2: <American Power Conversion> at usbus0
em1: link state changed to UP
WARNING: attempt to domain_add(netgraph) after domainfinalize()
em1: promiscuous mode enabled