Apparently spurious ZFS CRC errors (was Re: ZFS data error without reasons)

[Alexander Leidinger]

Quoting Mark Powell <M.S.Powell at salford.ac.uk> (from Wed, 25 Mar 2009  
10:45:43 +0000 (GMT)):

> On Wed, 25 Mar 2009, Alexander Leidinger wrote:
>
>> Quoting Mark Powell <M.S.Powell at salford.ac.uk> (from Fri, 20 Mar  
>> 2009 15:30:11 +0000 (GMT)):
>>> Hmmm. Perhaps I'm not being fair on 8. Just had a look at my  
>>> loader.conf for 7 and I can see that I used to run with every  
>>> zfs*disable on. I've just rebooted 8 with:
>>>
>>> vfs.zfs.cache_flush_disable: 1
>>> vfs.zfs.mdcomp_disable: 1
>>> vfs.zfs.prefetch_disable: 1
>>> vfs.zfs.zil_disable: 1
>>> hw.ata.wc: 1
>>>
>>> The current fs which produced errors on every scrub now reports no errors.
>>> I now need to find which option fixed it.
>>> I suspect hw.ata.wc. Is this still a known issue?
>
> Alex,
>   Thanks for the input,
>
>> I would expect that it is the combination of cache_flush_disable  
>> and zil_disable with the wc enable. If you reenable the zil and the  
>> cache flush, the wc should not cause the problems you see. You may  
>> want to have a look at  
>> http://www.solarisinternals.com/wiki/index.php/ZFS_Evil_Tuning_Guide for a  
>> more detailed description of what those options are doing (and why  
>> you should not disable those features on normal disks). I also  
>> suggest to not disable the meta-data compression, as it seems it  
>> only affects a small amount of meta-data instead of all meta-data.
>
> I initially ran 8 with default options i.e. write caching on, all  
> zfs options left enabled. I got the errors. Only then did I try  
> changing options, by turning off wc and disabling zfs options.
>   It's a little curious that I should reenable the wc, and all zfs  
> options except prefetch_disable. That takes me back to how I  
> originally started, but with the solution to the problem therefore  
> being disabling the prefetch.
>   Can prefetch really cause these problems? And if so why?

I don't think so. I missed the part where you explained this before.  
In this case it's really the write cache. The interesting questions is  
if this is because of the harddisks you use, or because of a bug in  
the software.

You run a very recent current? 1-2 weeks before there was a bug (not  
in ZFS) which caused CRC errors, but it was fixed shortly after it was  
noticed. If you haven't updated your system, it may be best to update  
it and try again. Please report back.

>> If you want to get more out of zfs, maybe vfs.zfs.vdev.max_pending  
>> could help if you are using SATA (as I read the zfs tuning guide,  
>> it makes sense to have a high value when you have command queueing,  
>> which we have with SCSI drives, but not yet with SATA drives and  
>> probably not at all with PATA drives).
>
> I'm running completely SATA with NCQ supporting drives. However, and  
> possibly as you say, NCQ is not really/properly supported in FBSD?

NCQ is not supported yet in FreeBSD. Alexander Motin said he is  
interested in implementing it, but I don't know about the status of  
this.

Bye,
Alexander.

-- 
Your business will assume vast proportions.

http://www.Leidinger.net    Alexander @ Leidinger.net: PGP ID = B0063FE7
http://www.FreeBSD.org       netchild @ FreeBSD.org  : PGP ID = 72077137