Zpool import failure, metadata checksum fails 8.0-RELEASE

Joseph Lenox lordofhyphens at gmail.com
Fri May 21 06:12:06 UTC 2010 

Update --

Updated to 8.0-STABLE (tagged 8.1-PRERELEASE), zpool is sitting at 
tx->tx state (according to top).

The root cause of this was apparently an enabled write cache on the sata 
controller (an adaptec model, can handle its own RAID5)+inopportune 
power failure; I've disabled it going forward.

I've done some searching through the 'net and haven't found any useful 
info for this type of situation--apparently Solaris just flags the pool 
as having soft errors and the user is expected to move on from there. As 
far as I can tell, the pool MUST be imported to perform any other kind 
of operation on it--you can't even destroy the pool. I don't know if 
recreating the pool from the underlying system destroys the data.

I'm going to leave it for 24 hours and double check to make sure the 
process is truly stuck (although I'm pretty sure it is). The current 
system install is a rebuild on different media--I do have the original 
HDD with the OS install on it, but any attempts to access the pool on 
that drive gives the same symptoms.

--Joseph Lenox

Command:
/sbin/zpool import -p failmode=panic -f valkyrie

PID   username THR PRI NICE SIZE       RES     STATE C TIME WCPU COMMAND
#### root          1      44   0        15668K  1936K tx->tx   1  0:00   
0.00%   zpool


/var/log/messages excerpt:

May 21 00:58:31 silmeria root: ZFS: checksum mismatch, zpool=valkyrie 
path=/dev/aacd3 offset=494926778368 size=1024
May 21 00:58:31 silmeria root: ZFS: checksum mismatch, zpool=valkyrie 
path=/dev/aacd1 offset=494926778368 size=1024
May 21 00:58:31 silmeria root: ZFS: checksum mismatch, zpool=valkyrie 
path=/dev/aacd4 offset=494926778368 size=1024
May 21 00:58:31 silmeria root: ZFS: checksum mismatch, zpool=valkyrie 
path=/dev/aacd5 offset=494926778368 size=1024
May 21 00:58:31 silmeria root: ZFS: checksum mismatch, zpool=valkyrie 
path=/dev/aacd0 offset=494926778880 size=512
May 21 00:58:31 silmeria root: ZFS: checksum mismatch, zpool=valkyrie 
path=/dev/aacd2 offset=494926778880 size=512
May 21 00:58:31 silmeria root: ZFS: checksum mismatch, zpool=valkyrie 
path=/dev/aacd0 offset=129568120320 size=1024
May 21 00:58:31 silmeria root: ZFS: checksum mismatch, zpool=valkyrie 
path=/dev/aacd2 offset=129568120320 size=1024
May 21 00:58:31 silmeria root: ZFS: checksum mismatch, zpool=valkyrie 
path=/dev/aacd1 offset=129568120320 size=1024
May 21 00:58:31 silmeria root: ZFS: checksum mismatch, zpool=valkyrie 
path=/dev/aacd3 offset=129568120320 size=1024
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie 
path=/dev/aacd4 offset=129568120320 size=512
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie 
path=/dev/aacd5 offset=129568120320 size=512
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie 
path=/dev/aacd4 offset=289968832000 size=1024
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie 
path=/dev/aacd5 offset=289968832000 size=1024
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie 
path=/dev/aacd0 offset=289968832512 size=1024
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie 
path=/dev/aacd2 offset=289968832512 size=1024
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie 
path=/dev/aacd1 offset=289968832512 size=512
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie 
path=/dev/aacd3 offset=289968832512 size=512
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie 
path=/dev/aacd3 offset=494926778368 size=1024
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie 
path=/dev/aacd1 offset=494926778368 size=1024
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie 
path=/dev/aacd4 offset=494926778368 size=1024
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie 
path=/dev/aacd5 offset=494926778368 size=1024
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie 
path=/dev/aacd0 offset=494926778880 size=512
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie 
path=/dev/aacd2 offset=494926778880 size=512
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie 
path=/dev/aacd0 offset=129568120320 size=1024
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie 
path=/dev/aacd2 offset=129568120320 size=1024
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie 
path=/dev/aacd1 offset=129568120320 size=1024
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie 
path=/dev/aacd3 offset=129568120320 size=1024
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie 
path=/dev/aacd4 offset=129568120320 size=512
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie 
path=/dev/aacd5 offset=129568120320 size=512
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie 
path=/dev/aacd4 offset=289968832000 size=1024
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie 
path=/dev/aacd5 offset=289968832000 size=1024
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie 
path=/dev/aacd0 offset=289968832512 size=1024
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie 
path=/dev/aacd2 offset=289968832512 size=1024
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie 
path=/dev/aacd1 offset=289968832512 size=512
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie 
path=/dev/aacd3 offset=289968832512 size=512
May 21 00:58:32 silmeria root: ZFS: zpool I/O failure, zpool=valkyrie 
error=86
May 21 00:58:32 silmeria root: ZFS: vdev I/O failure, zpool=valkyrie 
path= offset= size= error=

On 5/20/2010 1:18 AM, Joseph Lenox wrote:
> Hello, all--
>
> An unexpected powercycle apparently introduced (thanks to my system's 
> RAID controller) metadata checksum errors on the system. Attempts to 
> import that pool on any system hangs the command (such that it cannot 
> even be killed).
>
> I tried pulling out the OpenSolaris (2009.07) cd that I had on-hand, 
> but the livecd couldn't find any of the pools in the system--so no 
> help there.
>
> I'm certain a scrub will fix the metadata problems (with or without 
> dataloss, the data isn't critical enough to worry about losing a file 
> or ten -- losing everything is more of a pain).
>
> Anyone have any ideas for how I can get this pool fixed? I'm working 
> on getting 8.0-STABLE sources downloaded (via cvsup) for testing.
>
> --Joseph Lenox

More information about the freebsd-questions mailing list