ZFS Checksum errors

[Rich]

To be honest, if ZFS says you've got a ton of checksum errors, I would
strongly bet in favor of your data being damaged over a bug in ZFS.

What're the underlying disks and RAID card?

- Rich

On Thu, Jun 21, 2012 at 5:48 PM,  <rondzierwa at comcast.net> wrote:
>
> ok, i ran a verify on the raid, and it completed, so I believe that, from the hardware standpoint, da0 should be a functioning, 12TB disk.
>
> i did a zpool clear and re-ran the scrub, and the results were almost identical:
>
> phoenix# zpool status -v zfsPool
> pool: zfsPool
> state: ONLINE
> status: One or more devices has experienced an error resulting in data
> corruption. Applications may be affected.
> action: Restore the file in question if possible. Otherwise restore the
> entire pool from backup.
> see: http://www.sun.com/msg/ZFS-8000-8A
> scrub: scrub completed after 3h39m with 6353 errors on Thu Jun 21 17:28:10 2012
> config:
>
> NAME STATE READ WRITE CKSUM
> zfsPool ONLINE 0 0 6.20K
> da0 ONLINE 0 0 12.5K 24K repaired
>
> errors: Permanent errors have been detected in the following files:
>
> zfsPool/raid:<0x9e241>
> zfsPool/Build:<0x0>
> phoenix#
>
> along with the 6,353 I/O errors, there were over 12,000 checksum mismatch errors on the console.
>
>
> The recommendation from ZFS is to restore the file in question. At this point, I would just like to delete the two files.
> how do i do that?
>
> its these kind of antics that make me resistant to the thought of allowing ZFS to manage the raid. it seems to be having problems just managing a big file system. I don't want it to correct anything, or restore anything, just let me delete the files that hurt, fix up the free space list so it doesn't point outside the bounds of the disk, and get on with life.
>
> if its finding corrupted files that appear to not have a directory entry associated with them (unlinked files), why doesn't it just delete them? fsck asks you if you want to delete unlinked files, why doesn't zfs do the same, or at least give you the option of deleting bad files when it finds them?
>
> this is causing a lot of down time, and its making linux look very attractive in my organization. how do I get this untangled short of reformatting and starting over?
>
> ron.
>
>
> ----- Original Message -----
> From: "Xin LI" <delphij at gmail.com>
> To: rondzierwa at comcast.net
> Cc: "Steven Hartland" <killing at multiplay.co.uk>, freebsd-fs at freebsd.org
> Sent: Wednesday, June 20, 2012 6:56:09 PM
> Subject: Re: ZFS Checksum errors
>
> On Wed, Jun 20, 2012 at 1:55 PM, <rondzierwa at comcast.net> wrote:
>> Steve.
>>
>> well, it got done, and it found another anonymous file with errors . any idea how to get rid of these?
>
> Normally you need to "zpool clear zfsPool", and rerun zpool scrub. If
> you see these numbers growing again, it's likely that there are some
> other problems with your hardware. The recommended configuration is
> to use ZFS to manage disks, or at least split your RAID volumes into
> smaller ones by the way, since otherwise the volume is seen as a
> "single disk" to ZFS, making it impossible to repair data errors
> unless you add additional redundancy (zfs set copies=2, etc).
>
>>
>> thanks,
>> ron.
>>
>>
>>
>> phoenix# zpool status -v zfsPool
>> pool: zfsPool
>> state: ONLINE
>> status: One or more devices has experienced an error resulting in data
>> corruption. Applications may be affected.
>> action: Restore the file in question if possible. Otherwise restore the
>> entire pool from backup.
>> see: http://www.sun.com/msg/ZFS-8000-8A
>> scrub: scrub completed after 8h29m with 6276 errors on Wed Jun 20 16:18:01 2012
>> config:
>>
>> NAME STATE READ WRITE CKSUM
>> zfsPool ONLINE 0 0 6.17K
>> da0 ONLINE 0 0 13.0K 1.34M repaired
>>
>> errors: Permanent errors have been detected in the following files:
>>
>> zfsPool/raid:<0x9e241>
>> zfsPool/Build:<0x0>
>> phoenix#
>>
>>
>>
>>
>> ----- Original Message -----
>> From: "Steven Hartland" <killing at multiplay.co.uk>
>> To: rondzierwa at comcast.net, freebsd-fs at freebsd.org
>> Sent: Wednesday, June 20, 2012 1:58:20 PM
>> Subject: Re: ZFS Checksum errors
>>
>> ----- Original Message -----
>> From: <rondzierwa at comcast.net>
>> ..
>>
>>> zpool status indicates that a file has errors, but doesn't tell me its name:
>>>
>>> phoenix# zpool status -v zfsPool
>>> pool: zfsPool
>>> state: ONLINE
>>> status: One or more devices has experienced an error resulting in data
>>> corruption. Applications may be affected.
>>> action: Restore the file in question if possible. Otherwise restore the
>>> entire pool from backup.
>>> see: http://www.sun.com/msg/ZFS-8000-8A
>>> scrub: scrub in progress for 5h27m, 18.71% done, 23h42m to go
>>
>> Try waiting for the scrub to complete and see if its more helpful after that.
>>
>> Regards
>> Steve
>>
>> ================================================
>> This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it.
>>
>> In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337
>> or return the E.mail to postmaster at multiplay.co.uk.
>>
>> _______________________________________________
>> freebsd-fs at freebsd.org mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-fs
>> To unsubscribe, send any mail to "freebsd-fs-unsubscribe at freebsd.org"
>
>
>
> --
> Xin LI <delphij at delphij.net> https://www.delphij.net/
> FreeBSD - The Power to Serve! Live free or die
> _______________________________________________
> freebsd-fs at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-fs
> To unsubscribe, send any mail to "freebsd-fs-unsubscribe at freebsd.org"