misc/146786: zpool import hangs with checksum errors

[Joseph Lenox]

>Number:         146786
>Category:       misc
>Synopsis:       zpool import hangs with checksum errors
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Fri May 21 06:30:05 UTC 2010
>Closed-Date:
>Last-Modified:
>Originator:     Joseph Lenox
>Release:        8.0-STABLE
>Organization:
>Environment:
FreeBSD silmeria.valkyrie.dynalias.org 8.1-PRERELEASE FreeBSD 8.1-PRERELEASE #0: Thu May 20 21:29:45 CDT 2010     root at silmeria.valkyrie.dynalias.org:/usr/obj/usr/src/sys/ALTQ  amd64

>Description:
zpool is hanging (top status: tx->tx) indefinitely when attempting to import a pool containing zfs checksum mismatches. zpool cannot be killed, and a reboot of the system is required to access any zfs commands (even for an unaffected pool). 

This problem is evident on 8.0-RELEASE and 8.1-PRERELEASE.

Command used:

/sbin/zpool import -p failmode=panic -f valkyrie

/var/log/messages excerpt:
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd3 offset=494926778368 size=1024
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd1 offset=494926778368 size=1024
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd4 offset=494926778368 size=1024
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd5 offset=494926778368 size=1024
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd0 offset=494926778880 size=512
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd2 offset=494926778880 size=512
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd0 offset=129568120320 size=1024
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd2 offset=129568120320 size=1024
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd1 offset=129568120320 size=1024
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd3 offset=129568120320 size=1024
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd4 offset=129568120320 size=512
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd5 offset=129568120320 size=512
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd4 offset=289968832000 size=1024
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd5 offset=289968832000 size=1024
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd0 offset=289968832512 size=1024
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd2 offset=289968832512 size=1024
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd1 offset=289968832512 size=512
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd3 offset=289968832512 size=512
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd3 offset=494926778368 size=1024
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd1 offset=494926778368 size=1024
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd4 offset=494926778368 size=1024
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd5 offset=494926778368 size=1024
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd0 offset=494926778880 size=512
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd2 offset=494926778880 size=512
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd0 offset=129568120320 size=1024
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd2 offset=129568120320 size=1024
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd1 offset=129568120320 size=1024
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd3 offset=129568120320 size=1024
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd4 offset=129568120320 size=512
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd5 offset=129568120320 size=512
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd4 offset=289968832000 size=1024
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd5 offset=289968832000 size=1024
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd0 offset=289968832512 size=1024
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd2 offset=289968832512 size=1024
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd1 offset=289968832512 size=512
May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd3 offset=289968832512 size=512
May 21 00:58:32 silmeria root: ZFS: zpool I/O failure, zpool=valkyrie error=86
May 21 00:58:32 silmeria root: ZFS: vdev I/O failure, zpool=valkyrie path= offset= size= error=
>How-To-Repeat:
Invoke a checksum error in a ZFS dataset. This can be done with an enabled write cache for a RAID controller and inopportune powercycling.

Any attempt to access the pool in that system will cause checksum mismatch errors to be reported in /var/log/messages.
>Fix:


>Release-Note:
>Audit-Trail:
>Unformatted: