HEADS UP: ZFS v28 merged to 8-STABLE

[David Magda]

On Jun 10, 2011, at 17:25, Volodymyr Kostyrko wrote:

> Am I missing something? How about using fletcher[24] for dedup?

Fletcher is fairly weak as things go, and so even though two checksums are the same, there's a decent chance that the data is actually different. At least with recent releases of (Open)Solaris, when you enable do a 'dedup=on' the has used is SHA-256, which has very, very, very, low odds of having the same value occur from two different blocks of data.

When ZFS dedupe originally came out you could have one of the following values:
	. off
	. on (== sha256)
	. flecther4 with verify/compare
	. sha256 (without verify/compare)
	. sha256 with verify

There was a long-ish thread on zfs-discuss  fairly recently on whether SHA-256 was "good enough" where you could trust it, or whether one should do a verify step in addition to SHA-256:

	http://mail.opensolaris.org/pipermail/zfs-discuss/2011-January/046875.html

While some people argued that it was prudent to use "verify" (especially with your data/job on the line), a good portion of folks though said that it's not worth it (i.e., if you're not worried about being hit by lightning (2^-17 to 2^-18), you shouldn't be worried about a hash collision (2^-128)).