gvinum raid5 vs. ZFS raidz

[Scott Bennett]

Arthur Chance <freebsd at qeng-ho.org> wrote:

> On 06/08/2014 06:56, Scott Bennett wrote:
> > Arthur Chance <freebsd at qeng-ho.org> wrote:
> >> 
> >> [stuff deleted --SB]
> >       I wonder if what varies is the amount of space taken up by the
> > checksums.  If there's a checksum for each block, then the block size
> > would change the fraction of the space lost to checksums, and the parity
> > for the checksums would thus also change.  Enough to matter?  Maybe.
>
> I'm not a file system guru, but my (high level) understanding is as 
> follows. Corrections from anyone more knowledgeable welcome.
>
> 1. UFS and ZFS both use tree structures to represent files, with the 
> data stored at the leaves and bookkeeping stored in the higher nodes. 
> Therefore the overhead scales as the log of the data size, which is a 
> negligible fraction for any sufficiently large amount of data.
>
> 2. UFS doesn't have data checksums, it relies purely on the hardware 
> checksums. (This is the area I'm least certain of.)

     What hardware checksums are there?  I wasn't aware that this sort of
hardware kept any.
>
> 3. ZFS keeps its checksums in a Merkel tree 
> (http://en.wikipedia.org/wiki/Merkle_tree) so the checksums are held in 
> the bookkeeping blocks, not in the data blocks. This simply changes the 
> constant multiplier in front of the logarithm for the overhead. Also, I 
> believe ZFS doesn't use fixed size data blocks, but aggregates writes 
> into blocks of up to 128K.
>
> Personally, I don't worry about the overheads of checksumming as the 
> cost of the parity stripe(s) in raidz is dominant. It's a cost well 
> worth paying though - I have a 3 disk raidz1 pool and a disk went bad 
> within 3 months of building it (the manufacturer turned out to be having 
> a few problems at the time) but I didn't lose a byte.
>
     Good testimonial.  I'm not worried about the checksum space either.
I figure the benefits make it cheap at the price.  Of more concern to me
now is how I'm going to come up with at least two more 2 TB drives to set
up a raidz2 with a tolerably small fraction of the total space being tied
up in combined ZFS overhead (i.e., bookkeeping, parity, checksums, etc.)


                                  Scott Bennett, Comm. ASMELG, CFIAG
**********************************************************************
* Internet:   bennett at sdf.org   *xor*   bennett at freeshell.org  *
*--------------------------------------------------------------------*
* "A well regulated and disciplined militia, is at all times a good  *
* objection to the introduction of that bane of all free governments *
* -- a standing army."                                               *
*    -- Gov. John Hancock, New York Journal, 28 January 1790         *
**********************************************************************