8.1-RELEASE: ZFS data errors

[Mike Carlson]

On 11/09/2010 09:37 AM, Mike Carlson wrote:
> On 11/09/2010 03:23 AM, Ivan Voras wrote:
>> On 11/08/10 19:32, Mike Carlson wrote:
>>
>>> As soon as I create the volume and write data to it, it is reported as
>>> being corrupted:
>>>
>>>      write# zpool create filevol001 da2 da3 da4 da5 da6 da7 da8
>>> However, if I create a 'raidz' volume, no errors occur:
>> A very interesting problem. Can you check with some other kind of volume
>> manager that striping the data doesn't cause some unusual hardware
>> interaction? Can you try, as an experiment, striping them all with
>> gstripe (but you'll have to use a small stripe size like 16 KiB or 8 KiB)?
>>
>> _______________________________________________
>> freebsd-fs at freebsd.org mailing list
>> http://BLOCKEDBLOCKEDlists.freebsd.org/mailman/listinfo/freebsd-fs
>> To unsubscribe, send any mail to "freebsd-fs-unsubscribe at freebsd.org"
>>
> Sure:
>
>      write# gstripe label -v -s 16384  data /dev/da2 /dev/da3 /dev/da4
>      /dev/da5 /dev/da6 /dev/da7 /dev/da8
>      Metadata value stored on /dev/da2.
>      Metadata value stored on /dev/da3.
>      Metadata value stored on /dev/da4.
>      Metadata value stored on /dev/da5.
>      Metadata value stored on /dev/da6.
>      Metadata value stored on /dev/da7.
>      Metadata value stored on /dev/da8.
>      Done.
>      write# newfs -O2 -U /dev/stripe/data
>      /dev/stripe/data: 133522760.0MB (273454612256 sectors) block size
>      16384, fragment size 2048
>           using 627697 cylinder groups of 212.72MB, 13614 blks, 6848 inodes.
>           with soft updates
>      super-block backups (for fsck -b #) at:
>      ...
>      write# mount /dev/stripe/data /mnt
>      write# df -h
>      Filesystem            Size    Used   Avail Capacity  Mounted on
>      /dev/da0s1a           1.7T     22G    1.6T     1%    /
>      devfs                 1.0K    1.0K      0B   100%    /dev
>      /dev/stripe/data      126T    4.0K    116T     0%    /mnt
>      write# cd /tmp
>      write# md5 /tmp/random.dat.1
>      MD5 (/tmp/random.dat.1) = f795fa09e1b0975c0da0ec6e49544a36
>      write# cp random.dat.1 /mnt/
>      write# cp /mnt/random.dat.1 /mnt/random.dat.2
>      write# cp /mnt/random.dat.1 /mnt/random.dat.3
>      write# cp /mnt/random.dat.1 /mnt/random.dat.4
>      write# cp /mnt/random.dat.1 /mnt/random.dat.5
>      write# cp /mnt/random.dat.1 /mnt/random.dat.6
>      write# md5 /mnt/*
>      MD5 (/mnt/random.dat.1) = f795fa09e1b0975c0da0ec6e49544a36
>      MD5 (/mnt/random.dat.2) = f795fa09e1b0975c0da0ec6e49544a36
>      MD5 (/mnt/random.dat.3) = f795fa09e1b0975c0da0ec6e49544a36
>      MD5 (/mnt/random.dat.4) = f795fa09e1b0975c0da0ec6e49544a36
>      MD5 (/mnt/random.dat.5) = f795fa09e1b0975c0da0ec6e49544a36
>      MD5 (/mnt/random.dat.6) = f795fa09e1b0975c0da0ec6e49544a36
>      write# fsck /mnt
>      fsck: Could not determine filesystem type
>      write# fsck_ufs  /mnt
>      ** /dev/stripe/data (NO WRITE)
>      ** Last Mounted on /mnt
>      ** Phase 1 - Check Blocks and Sizes
>      Segmentation fault
>
> So, the data appears to be okay, I wanted to run through a FSCK just to
> do it but that seg faulted. Otherwise, that data looks good.
>
> Question, why did you recommend using a smaller stripe size? Is that to
> ensure a sample 1GB test file gets written across ALL disk members?
> _______________________________________________
> freebsd-fs at freebsd.org mailing list
> http://BLOCKEDlists.freebsd.org/mailman/listinfo/freebsd-fs
> To unsubscribe, send any mail to "freebsd-fs-unsubscribe at freebsd.org"
>
Oh, I almost forgot, here is the ZFS version of that gstripe array:

    write# zpool create test01 /dev/stripe/data
    write# cp /tmp/random.dat.1 /test01/
    write# cp /test01/random.dat.1 /test01/random.dat.2
    write# cp /test01/random.dat.1 /test01/random.dat.3
    write# cp /test01/random.dat.1 /test01/random.dat.4
    write# cp /test01/random.dat.1 /test01/random.dat.5
    write# cp /test01/random.dat.1 /test01/random.dat.6
    write# md5 /test01/*
    MD5 (/test01/random.dat.1) = f795fa09e1b0975c0da0ec6e49544a36
    MD5 (/test01/random.dat.2) = f795fa09e1b0975c0da0ec6e49544a36
    MD5 (/test01/random.dat.3) = f795fa09e1b0975c0da0ec6e49544a36
    MD5 (/test01/random.dat.4) = f795fa09e1b0975c0da0ec6e49544a36
    MD5 (/test01/random.dat.5) = f795fa09e1b0975c0da0ec6e49544a36
    MD5 (/test01/random.dat.6) = f795fa09e1b0975c0da0ec6e49544a36
    write# zpool scrub
    write# zpool status
       pool: test01
      state: ONLINE
      scrub: scrub completed after 0h0m with 0 errors on Tue Nov  9
    09:41:34 2010
    config:

         NAME           STATE     READ WRITE CKSUM
         test01         ONLINE       0     0     0
           stripe/data  ONLINE       0     0     0

    errors: No known data errors

Again, no errors.