gvinum raid5 vs. ZFS raidz

[Trond Endrestøl]

On Thu, 7 Aug 2014 03:31-0500, Scott Bennett wrote:

> Arthur Chance <freebsd at qeng-ho.org> wrote:
> 
> > On 06/08/2014 06:56, Scott Bennett wrote:
> > > Arthur Chance <freebsd at qeng-ho.org> wrote:
> > >> 
> > >> [stuff deleted --SB]
> > >       I wonder if what varies is the amount of space taken up by the
> > > checksums.  If there's a checksum for each block, then the block size
> > > would change the fraction of the space lost to checksums, and the parity
> > > for the checksums would thus also change.  Enough to matter?  Maybe.
> >
> > I'm not a file system guru, but my (high level) understanding is as 
> > follows. Corrections from anyone more knowledgeable welcome.
> >
> > 1. UFS and ZFS both use tree structures to represent files, with the 
> > data stored at the leaves and bookkeeping stored in the higher nodes. 
> > Therefore the overhead scales as the log of the data size, which is a 
> > negligible fraction for any sufficiently large amount of data.
> >
> > 2. UFS doesn't have data checksums, it relies purely on the hardware 
> > checksums. (This is the area I'm least certain of.)
> 
>      What hardware checksums are there?  I wasn't aware that this sort of
> hardware kept any.

To quote http://en.wikipedia.org/wiki/Disk_sector:

In disk drives, each physical sector is made up of three basic parts, 
the sector header, the data area and the error-correcting code (ECC).

> > 3. ZFS keeps its checksums in a Merkel tree 
> > (http://en.wikipedia.org/wiki/Merkle_tree) so the checksums are held in 
> > the bookkeeping blocks, not in the data blocks. This simply changes the 
> > constant multiplier in front of the logarithm for the overhead. Also, I 
> > believe ZFS doesn't use fixed size data blocks, but aggregates writes 
> > into blocks of up to 128K.
> >
> > Personally, I don't worry about the overheads of checksumming as the 
> > cost of the parity stripe(s) in raidz is dominant. It's a cost well 
> > worth paying though - I have a 3 disk raidz1 pool and a disk went bad 
> > within 3 months of building it (the manufacturer turned out to be having 
> > a few problems at the time) but I didn't lose a byte.
> >
>      Good testimonial.  I'm not worried about the checksum space either.
> I figure the benefits make it cheap at the price.  Of more concern to me
> now is how I'm going to come up with at least two more 2 TB drives to set
> up a raidz2 with a tolerably small fraction of the total space being tied
> up in combined ZFS overhead (i.e., bookkeeping, parity, checksums, etc.)
> 
> 
>                                   Scott Bennett, Comm. ASMELG, CFIAG
> **********************************************************************
> * Internet:   bennett at sdf.org   *xor*   bennett at freeshell.org  *
> *--------------------------------------------------------------------*
> * "A well regulated and disciplined militia, is at all times a good  *
> * objection to the introduction of that bane of all free governments *
> * -- a standing army."                                               *
> *    -- Gov. John Hancock, New York Journal, 28 January 1790         *
> **********************************************************************

-- 
+-------------------------------+------------------------------------+
| Vennlig hilsen,               | Best regards,                      |
| Trond Endrestøl,              | Trond Endrestøl,                   |
| IT-ansvarlig,                 | System administrator,              |
| Fagskolen Innlandet,          | Gjøvik Technical College, Norway,  |
| tlf. mob.   952 62 567,       | Cellular...: +47 952 62 567,       |
| sentralbord 61 14 54 00.      | Switchboard: +47 61 14 54 00.      |
+-------------------------------+------------------------------------+