zfs mirror: 1 disk lost, corrupted other disk. crashes zfs tools
and panics system
Edward Sutton
mirror176 at hotmail.com
Wed Jun 8 22:44:44 UTC 2011
As a disk started clicking but running slow, then died out, trying to reboot the system ended up in a crash early in the boot sequence; not sure if it was zfs related or not.
I managed to get back most data from the pool. It ends up with problems for some filesystems created under /var. Created per instructions on http://wiki.freebsd.org/RootOnZFS/GPTZFSBoot/Mirror
I then cloned the disk for further & safer play which appears as:
pool: zroot
state: FAULTED
status: An intent log record could not be read.
Waiting for adminstrator intervention to fix the faulted pool.
action: Either restore the affected device(s) and run 'zpool online',
or ignore the intent log records by running 'zpool clear'.
see: http://www.sun.com/msg/ZFS-8000-K4
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
zroot FAULTED 0 0 0 bad intent log
mirror DEGRADED 0 0 0
gpt/disk0 ONLINE 0 0 0
gpt/disk1 UNAVAIL 0 0 0 cannot open
Going by my memory here: It fails to boot with "ROOT MOUNT ERROR". I had luck with `zpool clear, removing disk1, and mounting all pool partitions. Data could be copied off but some filesystems were missing and attempts to list all snapshots segfaults. Every export/import requires a zpool clear. Attempting a scrub lead to a panic on every import on the pool thereafter. Tried a -current snapshot (FreeBSD-9.0-CURRENT-201105-amd64-dvd1.iso) livecd mode with similar panic results to 8.2-release.
Seems the disk already imported after last restore from its dd backup. Steps I heard of to get to more data involve using zdb to locate IDs (from -dddd?) but more than just `zdb` ends in an error such as the following run.
zdb zroot
version=13
name='zroot'
state=0
txg=2181411
pool_guid=8313451715893809405
hostid=1510368511
hostname=''
vdev_tree
type='root'
id=0
guid=8313451715893809405
children[0]
type='mirror'
id=0
guid=14331193696332474730
whole_disk=0
metaslab_array=23
metaslab_shift=31
ashift=9
asize=1996098895872
is_log=0
children[0]
type='disk'
id=0
guid=2590809366000450414
path='/dev/gpt/disk0'
whole_disk=0
DTL=29
children[1]
type='disk'
id=1
guid=13294596440516983442
path='/dev/gpt/disk1'
whole_disk=0
DTL=631
Assertion failed: (doi.doi_type == DMU_OT_DSL_DIR (0x8 == 0xc)), file /usr/src/cddl/lib/libzpool/../../../sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_dir.c, line 93.
Abort (core dumped)
Any suggestions to try to get the data back? If I can provide any useful details to get panics eliminated, I'd be glad to provide them but do not know how to start. I do have some crash dumps from may 25-31. Not sure which of the dumps contain useful data now or that they are complete; attempted scrub+dump on boot was going very slow so I used ctrl + C at least once.
More information about the freebsd-fs
mailing list