zfs mirror: 1 disk lost, corrupted other disk. crashes zfs tools and panics system

Edward Sutton mirror176 at hotmail.com
Wed Jun 8 22:44:44 UTC 2011


As a disk started clicking but running slow, then died out, trying to reboot the system ended up in a crash early in the boot sequence; not sure if it was zfs related or not. 
I managed to get back most data from the pool. It ends up with problems for some filesystems created under /var. Created per instructions on http://wiki.freebsd.org/RootOnZFS/GPTZFSBoot/Mirror
I then cloned the disk for further & safer play which appears as:

  pool: zroot
 state: FAULTED
status: An intent log record could not be read.
        Waiting for adminstrator intervention to fix the faulted pool.
action: Either restore the affected device(s) and run 'zpool online',
        or ignore the intent log records by running 'zpool clear'.
   see: http://www.sun.com/msg/ZFS-8000-K4
 scrub: none requested
config:

        NAME           STATE     READ WRITE CKSUM
        zroot          FAULTED      0     0     0  bad intent log
          mirror       DEGRADED     0     0     0
            gpt/disk0  ONLINE       0     0     0
            gpt/disk1  UNAVAIL      0     0     0  cannot open

Going by my memory here: It fails to boot with "ROOT MOUNT ERROR". I had luck with `zpool clear, removing disk1, and mounting all pool partitions. Data could be copied off but some filesystems were missing and attempts to list all snapshots segfaults. Every export/import requires a zpool clear. Attempting a scrub lead to a panic on every import on the pool thereafter. Tried a -current snapshot (FreeBSD-9.0-CURRENT-201105-amd64-dvd1.iso) livecd mode with similar panic results to 8.2-release.

Seems the disk already imported after last restore from its dd backup. Steps I heard of to get to more data involve using zdb to locate IDs (from -dddd?) but more than just `zdb` ends in an error such as the following run.

zdb  zroot
    version=13
    name='zroot'
    state=0
    txg=2181411
    pool_guid=8313451715893809405
    hostid=1510368511
    hostname=''
    vdev_tree
        type='root'
        id=0
        guid=8313451715893809405
        children[0]
                type='mirror'
                id=0
                guid=14331193696332474730
                whole_disk=0
                metaslab_array=23
                metaslab_shift=31
                ashift=9
                asize=1996098895872
                is_log=0
                children[0]
                        type='disk'
                        id=0
                        guid=2590809366000450414
                        path='/dev/gpt/disk0'
                        whole_disk=0
                        DTL=29
                children[1]
                        type='disk'
                        id=1
                        guid=13294596440516983442
                        path='/dev/gpt/disk1'
                        whole_disk=0
                        DTL=631
Assertion failed: (doi.doi_type == DMU_OT_DSL_DIR (0x8 == 0xc)), file /usr/src/cddl/lib/libzpool/../../../sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_dir.c, line 93.
Abort (core dumped)

Any suggestions to try to get the data back? If I can provide any useful details to get panics eliminated, I'd be glad to provide them but do not know how to start. I do have some crash dumps from may 25-31. Not sure which of the dumps contain useful data now or that they are complete; attempted scrub+dump on boot was going very slow so I used ctrl + C at least once.
 		 	   		  


More information about the freebsd-fs mailing list