ZFS RaidZ2 with 24 drives?

[Thomas Burgess]

2009/12/16 James R. Van Artsdalen <james-freebsd-fs2 at jrv.org>

> Matt Simerson wrote:
>
> >> In general, it's always best to let ZFS handle the raid stuff and not
> >> use the hardware raid settings.
> >
> > I'd like to see some evidence to back that statement up. The only time
> > I've seen better ZFS performance numbers than what I'm getting with
> > FreeBSD 8 ZFS + Areca RAID6 is when I tested OpenSolaris with them
> > Marvell SATA controllers. But that was in August of 2008, and ZFS on
> > FreeBSD performs much better now. Some updated benchmarks would be
> > welcome.
>
> You're fixating on throughput.  Moreover, Marvell is the wrong
> controller for FreeBSD 8: use the SIIS driver with a controller based on
> the Silicon Image 3124.
>
> Doing redundancy in ZFS rather than the controller has two obvious
> advantages:
>
> 1. ZFS has access the filesystem checksums, signatures and txtags
> whereas RAID controllers do not.  ZFS can always tell when data recovery
> is needed, which copy of the data is correct in a mirror, and which disk
> in a striped-parity set needs reconstruction.  And ZFS can tell if no
> recovery was possible.  A RAID controller can't always do this right.
>
> 2. More important, by letting ZFS handle RAID you can spread the risk
> across controllers and drivers as well as disks.
>
> For example, my home pool is an array of MIRRORs, each MIRROR carefully
> arranged so that if one component is a Seagate disk, the other is *not*
> a Seagate.  For every mirror the two disks are in different enclosures
> (using different power supplies), each enclosure set connected to a
> different host controller.  And I'm about to put half of the disks in
> SAS enclosures connected to LSI 3801e controllers using the MPT driver:
> once done I'll be fully redundant with respect to disk drives,
> enclosures & disk power supplies, cables, controllers and drivers.
> _______________________________________________
> freebsd-fs at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-fs
> To unsubscribe, send any mail to "freebsd-fs-unsubscribe at freebsd.org"
>
>
exactly.  One thing most people don't know about hard drives in general is
that sometimes up to 30% of the space is actually ECC.  With software raid
systems like ZFS, this will eventually be somethign that we can take
advantage of.   The zfs checksums (only?) work best when you give each it
raw drives.  Because of this,  you can imagine a scenario where  allowing
ZFS to use this ECC space as raw storage, while leaving the data corrections
to ZFS would be ideal.  It's not only a matter of space, it will also lead
to nice improvements in speed.  (more data can be read/written by the head
as it passes)

Also, ZFS can take advantage of the drives write cache in ways that a
hardware raid controller cannot.  I'm not saying hardware raid doesn't
perform well, it does.  It's just a waste of money when you're using ZFS.
Your money is MUCH better spent on bigger drives, more ram and maybe an SSD
or two