ZFS fatal double fault upon reboot after upgrade to 10.2
Freek Dijkstra
public at macfreek.nl
Tue Oct 6 19:28:02 UTC 2015
Hi all,
After an upgrade from FreeBSD 10.0 to 10.2, my system no longer boots
with an error:
Trying to mount root from zfs:zroot []...
Fatal double fault:
When booting from an USB stick, I'm able to mount my zpool just fine, so
I suspect this is a software or configuration error, rather than a
hardware error.
I appreciate any advice how to proceed. Details of my case are below.
Context:
I have a system running FreeBSD as my router, since about 3 year.
Hardware: Soekris net6501-50 Intel Atom E640 CPU (32 bit), just 1 GByte
memory.
Disks: two SSD (mini-SATA) Intel SSD 525 Series 30GByte.
ZFS config:
pool: zroot
state: ONLINE
scan: resilvered 13.0M in 0h0m with 0 errors on Sat Oct 3 15:37:46 2015
config:
NAME STATE READ WRITE CKSUM
zroot ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
gpt/system0 ONLINE 0 0 0
gpt/system1 ONLINE 0 0 0
errors: No known data errors
System is running FreeBSD 10.0, so the upgrade to 10.2 is actually long
overdue.
`freebsd-update -r 10.2-RELEASE upgrade` worked fine, just had to
restart because of lost connection (broken pipe) and I forgot to use
`screen`.
/usr/sbin/freebsd-update install
Installing updates...^@^@^@^@^@^@^@^@^@^@^@^@^@
Kernel updates have been installed. Please reboot and run
"/usr/sbin/freebsd-update install" again to finish installing updates.
[root at wolfje] ~# /sbin/shutdown -r now
Shutdown NOW!
shutdown: [pid 70238]
Then after reboot, system did not come up.
Memory check (by Soekris BIOS) is fine.
Excerpt from the tty output:
/boot/kernel/kernel text=0xf79f95 data=0xdb7e8+0x160f88
syms=[0x4+0xd7fa0+0x4+0x15f0e8]
/boot/kernel/zfs.ko text=0x19267c data=0x9d24+0x546cc
syms=[0x4+0x113f0+0x4+0x181ed]
loading required module 'opensolaris'
/boot/kernel/opensolaris.ko text=0x1758 data=0x18c+0xc24
syms=[0x4+0x850+0x4+0x882]
/boot/kernel/if_tap.ko text=0x3cb8 data=0x2c4+0x38
syms=[0x4+0xac0+0x4+0xa6c]
/boot/kernel/ip_mroute.ko text=0x904c data=0x2cc+0x360c
syms=[0x4+0xe70+0x4+0x1014]
Booting...
ACPI BIOS Error (bug): A valid RSDP was not found (20150515/tbxfroot-258)
Copyright (c) 1992-2015 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 10.2-RELEASE #0 r286666: Wed Aug 12 19:31:38 UTC 2015
root at releng1.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC i386
FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512
CPU: Genuine Intel(R) CPU @ 1.00GHz (1000.02-MHz 686-class CPU)
Origin="GenuineIntel" Id=0x20661 Family=0x6 Model=0x26 Stepping=1
Features=0xbfe9fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
Features2=0x40e3bd<SSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,MOVBE>
AMD Features=0x20100000<NX,LM>
AMD Features2=0x1<LAHF>
VT-x: (disabled in BIOS) HLT,PAUSE
TSC: P-state invariant, performance statistics
real memory = 1073610752 (1023 MB)
avail memory = 1025433600 (977 MB)
MPTable: <Soekris net6501 >
[...]
ZFS NOTICE: Prefetch is disabled by default on i386 -- to enable,
add "vfs.zfs.prefetch_disable=0" to /boot/loader.conf.
ZFS WARNING: Recommended minimum kmem_size is 512MB; expect unstable
behavior.
Consider tuning vm.kmem_size and vm.kmem_size_max
in /boot/loader.conf.
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
[...]
ada0 at ahcich1 bus 0 scbus1 target 0 lun 0
uhub5: 1 port with 1 removable, self powered
ada0: <INTEL SSDMCEAC030B3 LLLi> ACS-2 ATA SATA 3.x device
ada0: Serial Number CVLI3212005D030H
ada0: 300.000MB/s transfersuhub6: 1 port with 1 removable, self powered
(SATA 2.x, UDMA6, PIO 8192bytes)
ada0: Command Queueing enabled
ada0: 28626MB (58626288 512 byte sectors: 16H 63S/T 16383C)
SMP: AP CPU #1 Launched!
Timecounter "TSC" frequency 1000022290 Hz quality 1000
Root mount waiting for: usbus7 usbus3
uhub3: 3 ports with 3 removable, self powered
uhub7: 3 ports with 3 removable, self powered
Trying to mount root from zfs:zroot []...
Fatal double fault:
eip = 0xc181f6d4
esp = 0xda377ff0
ebp = 0xda378024
cpuid = 0; apic id = 00
panic: double fault
cpuid = 0
KDB: stack backtrace:
#0 0xc0b720f2 at kdb_backtrace+0x52
#1 0xc0b332cb at vpanic+0x11b
#2 0xc0b331ab at panic+0x1b
#3 0xc10554eb at dblfault_handler+0xab
Uptime: 4s
Automatic reboot in 15 seconds - press a key on the console to abort
After a reboot, the same error occurs.
The eip, esp and ebp number are exactly the same for each reboot, so I
suspect a software issue (as opposed to a hardware issue).
The warning about kernel memory also occured with FreeBSD 10.0, but I
had ignored that since everything seemed to work in practice.
I wanted to check if the zpool is fine, so I booted the machine from an
USB stick with FreeBSD-10.2-RELEASE-i386-mini-memstick.img.
This worked fine. I could even mount my zpool:
# zpool list
ZFS NOTICE: Prefetch is disabled by default on i386 -- to enable,
add "vfs.zfs.prefetch_disable=0" to /boot/loader.conf.
ZFS WARNING: Recommended minimum kmem_size is 512MB; expect unstable
behavior.
Consider tuning vm.kmem_size and vm.kmem_size_max
in /boot/loader.conf.
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
no pools available
# zpool import
pool: zroot
id: 13393886111405456989
state: ONLINE
status: The pool was last accessed by another system.
action: The pool can be imported using its name or numeric identifier and
the '-f' flag.
see: http://illumos.org/msg/ZFS-8000-EY
config:
zroot ONLINE
mirror-0 ONLINE
gpt/system0 ONLINE
gpt/system1 ONLINE
# ls /mnt
# zpool import -f -o readonly=on -R /mnt zroot
# ls /mnt
.cshrc boot lib proc tmp
.profile dev libexec rescue usr
.rnd entropy man8 root var
COPYRIGHT etc media sbin
bin home mnt sys
How to proceed?
A search on the Internet for "ZFS fatal double fault" gave the following
result:
http://lists.freebsd.org/pipermail/freebsd-stable/2014-December
/081196.html
> Either switch to amd64 or increase the stack size of your kernel.
Would that help here as well? How do I increase the stack size of the
kernel?
Regards,
Freek Dijkstra
More information about the freebsd-questions
mailing list