ZFS scrub/selfheal not really working

[Nathanael Hoyle]

Dmitry Marakasov wrote:
> * Andrew Snow (andrew at modulus.org) wrote:
>
>   
>> Because your disk subsystem is broken and keeps returning new sets of 
>> bad sectors.
>>     
>
> Still running my utility fixed them. Running scrub don't.
>
>   

scrub isn't trying to repair bad sectors on your hard drive.  scrub is 
trying to restore the health of all copies of the data based on 
available parity or mirrors.  These are different goals.  As was 
suggested, I expect that if you have either power supply or controller 
errors, you are having new, unique, device-layer read errors every scrub 
pass (note your counts between passes are very different).  You are not 
having zpool-layer failures (zfs is successfully using parity or mirror 
information to ensure that no bad data was returned to the application 
layer).  In the course of a scrub, a read error should result in a new 
sector being written with the appropriate data, based on parity or 
mirror data, this does not prevent the original sector from having read 
failures if you attempt to read it on a block by block basis.  The 
diagram you referenced is too high level to be meaningful.  For better 
treatment, I suggest you refer to 
http://dlc.sun.com/pdf/819-5461/819-5461.pdf specifically "Checking ZFS 
Data Integrity" beginning on page 251, and continuing through say page 261.

Best of luck,
-Nathanael