ZFS or UFS for 4TB hardware RAID6?

[Richard Mahlerwein]

--- On Mon, 7/13/09, Maxim Khitrov <mkhitrov at gmail.com> wrote:

> From: Maxim Khitrov <mkhitrov at gmail.com>
> Subject: Re: ZFS or UFS for 4TB hardware RAID6?
> To: mahlerrd at yahoo.com
> Cc: "Free BSD Questions list" <freebsd-questions at freebsd.org>
> Date: Monday, July 13, 2009, 2:02 PM
> On Mon, Jul 13, 2009 at 1:46 PM,
> Richard Mahlerwein<mahlerrd at yahoo.com>
> wrote:
> >>
> >> Your mileage may vary, but...
> >>
> >> I would investigate either using more spindles if
> you want
> >> to stick to RAID6, or perhaps using another RAID
> level if
> >> you will be with 4 drives for a while.  The
> reasoning
> >> is that there's an overhead with RAID 6 - parity
> blocks are
> >> written to 2 disks, so in a 4 drive combination
> you have 2
> >> drives with data and 2 with parity.
> >>
> >> With 4 drives, you could get much, much higher
> performance
> >> out of RAID10 (which is alternatively called
> RAID0+1 or
> >> RAID1+0 depending on the manufacturer and on how
> accurate
> >> they wish to be, and on how they actually
> implemented it,
> >> too). This would also mean 2 usable drives, as
> well, so
> >> you'd have the same space available in RAID10 as
> your
> >> proposed RAID6.
> >>
> >> I would confirm you can, on the fly, convert from
> RAID10 to
> >> RAID6 after you add more drives.  If you can not,
> then
> >> by all means stick with RAID6 now!
> >>
> >> With 4 1 TB drives (for simpler examples)
> >> RAID5 = 3 TB available, 1 TB worth used in
> "parity".
> >> Fast reads, slow writes.
> >> RAID6 = 2 TB available, 2 TB worth used in
> "parity".
> >> Moderately fast reads, slow writes.
> >> RAID10 = 2 TB available, 2TB in duplicate copies
> (easier
> >> work than parity calculations).  Very fast
> reads,
> >> moderately fast writes.
> >>
> >> When you switch to, say, 8 drives, the numbers
> start to
> >> change a bit.
> >> RAID5 = 7TB available, 1 lost.
> >> RAID6 = 6TB available, 2 lost.
> >> RAID10 = 4TB available, 4 lost.
> >>
> >
> > Sorry, consider myself chastised for having missed the
> "Security is more important than performance" bit. I tend
> toward solutions that show the most value, and with 4
> drives, it seems that I'd stick with the same "data
> security" only pick up the free speed of RAID10.  Change
> when you get to 6 or more drives, if necessary.
> >
> > For data security, I can't answer for the UFS2 vs.
> ZFS.  For hardware setup, let me amend everything I said
> above with the following:
> >
> > Since you are seriously focusing on data integrity,
> ignore everything I said but make sure you have good
> backups!  :)
> >
> > Sorry,
> > -Rich
> 
> No problem :) I've been doing some reading since I posted
> this
> question and it turns out that the controller will actually
> not allow
> me to create a RAID6 array using only 4 drives. 3ware
> followed the
> same reasoning as you; with 4 drives use RAID10.
> 
> I know that you can migrate from one to the other when a
> 5th disk is
> added, but RAID10 can only handle 2 failed drives if they
> are from
> separate RAID1 groups. In this way, it is just slightly
> less resilient
> to failure than RAID6. With this new information, I think I
> may as
> well get one more 2TB drive and start with 6TB of RAID6
> space. This
> will be less of a headache later on.
> 
> - Max

Just as a question: how ARE you planning on backing this beast up?  While I don't want to sound like a worry-wort, I have had odd things happen at the worst of times.  RAID cards fail, power supplies let out the magic smoke, users delete items they really want back... *sigh*

A bit of reading shows that ZFS, if it's stable enough, has some really great features that would be nice on such a large pile o' drives.  

See http://wiki.freebsd.org/ZFSQuickStartGuide

I guess the last question I'll ask (as any more may uncover my ignorance) is if you need to use hardware RAID at all?  It seems both UFS2 and ZFS can do software RAID which seems to be quite reasonable with respect to performance and in many ways seems to be more robust since it is a bit more portable (no specialized hardware).

There are others who may respond with better information on that front.  I've been a strong proponent of hardware RAID, but have recently begun to realize many of the reasons for that are only of limited validity now.

-Rich