ZFS not returning correct data if checksum fails? (copies=2)

[Aravind K. Mikkilineni]

Hi,

I may have experienced some odd behavior from ZFS running 8.0-STABLE-201004 amd64.

I have a zpool 'zhome' on my laptop with checksum=on and copies=2. I was getting ready to burn a FreeBSD DVD today and was verifying the checkums of the image I downloaded (stored 
on zhome). I got a match on the md5 hash, but a mismatch on the sha256 hash. Rerunning the sha256 hash gave the correct result. I luckily just got in the office earlier and had a 
zpool status in my terminal buffer which indicated no errors on zhome. Running zpool status after the checksumming operations shows one checksum error. 

I was under the impression that ZFS, upon encountering a checksum error, would check the second copy of the block (since I have set copies=2) and send that data instead (assuming it 
was correct), that does not seem to have happened. I have run into checksum errors in the past where I had copies=1, in those cases ZFS would simply stop reading the file at that 
point and I would get a read error, which I did not in this case. Also I am fairly certain that the dvd image was not buffered in RAM between hashes (i.e. each hash read the data 
from disk).

Perhaps the checksum was correct the first time around and some other part of my system corrupted the data? RAM has been tested many months ago using memtest86 and several other 
tools without problem (not that new problems couldn't have popped up).

akm at moony:~$ zpool status zhome
  pool: zhome
 state: ONLINE
 scrub: none requested
config:

        NAME         STATE     READ WRITE CKSUM
        zhome        ONLINE       0     0     0
          ad4p3.eli  ONLINE       0     0     0

errors: No known data errors
akm at moony:ISOs$ grep *8.0*amd64-dvd1* CHECKSUM.* | cut -d\= -f2
 227adf4b4010aa1bc98baf50953ec2d0
 4c087a9d23a53f861f3548461cde500a6db709ff704e972c51677531ea6d65b3
akm at moony:ISOs$ md5 FreeBSD-8.0-STABLE-201005-amd64-dvd1.iso | cut -d\= -f2
 227adf4b4010aa1bc98baf50953ec2d0
akm at moony:ISOs$ sha256  FreeBSD-8.0-STABLE-201005-amd64-dvd1.iso | cut -d\= -f2
 4fcbcaacf14ff7c0394f119b1ca93e645456241d5b7d42aaf8f92b5a70b279d4
akm at moony:ISOs$ sha256  FreeBSD-8.0-STABLE-201005-amd64-dvd1.iso | cut -d\= -f2
 4c087a9d23a53f861f3548461cde500a6db709ff704e972c51677531ea6d65b3
akm at moony:ISOs$ zpool status zhome
  pool: zhome
 state: ONLINE
status: One or more devices has experienced an unrecoverable error.  An
        attempt was made to correct the error.  Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
        using 'zpool clear' or replace the device with 'zpool replace'.
   see: http://www.sun.com/msg/ZFS-8000-9P
 scrub: none requested
config:

        NAME         STATE     READ WRITE CKSUM
        zhome        ONLINE       0     0     0
          ad4p3.eli  ONLINE       0     0     1

errors: No known data errors
akm at moony:ISOs$ 

	-akm