HEADS UP: ZFS v28 merged to 8-STABLE
dmagda at ee.ryerson.ca
Sun Jun 12 04:37:17 UTC 2011
On Jun 10, 2011, at 17:25, Volodymyr Kostyrko wrote:
> Am I missing something? How about using fletcher for dedup?
Fletcher is fairly weak as things go, and so even though two checksums are the same, there's a decent chance that the data is actually different. At least with recent releases of (Open)Solaris, when you enable do a 'dedup=on' the has used is SHA-256, which has very, very, very, low odds of having the same value occur from two different blocks of data.
When ZFS dedupe originally came out you could have one of the following values:
. on (== sha256)
. flecther4 with verify/compare
. sha256 (without verify/compare)
. sha256 with verify
There was a long-ish thread on zfs-discuss fairly recently on whether SHA-256 was "good enough" where you could trust it, or whether one should do a verify step in addition to SHA-256:
While some people argued that it was prudent to use "verify" (especially with your data/job on the line), a good portion of folks though said that it's not worth it (i.e., if you're not worried about being hit by lightning (2^-17 to 2^-18), you shouldn't be worried about a hash collision (2^-128)).
More information about the freebsd-stable