ZFS or UFS for 4TB hardware RAID6?

[Richard Mahlerwein]

--- On Sun, 7/12/09, Maxim Khitrov <mkhitrov at gmail.com> wrote:

> From: Maxim Khitrov <mkhitrov at gmail.com>
> Subject: ZFS or UFS for 4TB hardware RAID6?
> To: "Free BSD Questions list" <freebsd-questions at freebsd.org>
> Date: Sunday, July 12, 2009, 11:47 PM
> Hello all,
> 
> I'm about to build a new file server using 3ware 9690SA-8E
> controller
> and 4x Western Digital RE4-GP 2TB drives in RAID6. It is
> likely to
> grow in the future up to 10TB. I may use FreeBSD 8 on this
> one, since
> the release will likely be made by the time this server
> goes into
> production. The question is a simple one - I have no
> experience with
> ZFS and so wanted to ask for recommendations of that versus
> UFS2. How
> stable is the implementation and does it offer any benefits
> in my
> setup (described below)?
> 
> All of the RAID6 space will only be used for file storage,
> accessible
> by network using NFS and SMB. It may be split into
> separate
> partitions, but most likely the entire array will be one
> giant storage
> area that is expanded every time another hard drive is
> added. The OS
> and all installed apps will be on a separate software RAID1
> array.
> 
> Given that security is more important than performance,
> what would be
> your recommended setup and why?
> 
> - Max

Your mileage may vary, but...

I would investigate either using more spindles if you want to stick to RAID6, or perhaps using another RAID level if you will be with 4 drives for a while.  The reasoning is that there's an overhead with RAID 6 - parity blocks are written to 2 disks, so in a 4 drive combination you have 2 drives with data and 2 with parity.  

With 4 drives, you could get much, much higher performance out of RAID10 (which is alternatively called RAID0+1 or RAID1+0 depending on the manufacturer and on how accurate they wish to be, and on how they actually implemented it, too). This would also mean 2 usable drives, as well, so you'd have the same space available in RAID10 as your proposed RAID6.  

I would confirm you can, on the fly, convert from RAID10 to RAID6 after you add more drives.  If you can not, then by all means stick with RAID6 now!

With 4 1 TB drives (for simpler examples)
RAID5 = 3 TB available, 1 TB worth used in "parity".  Fast reads, slow writes. 
RAID6 = 2 TB available, 2 TB worth used in "parity".  Moderately fast reads, slow writes.
RAID10 = 2 TB available, 2TB in duplicate copies (easier work than parity calculations).  Very fast reads, moderately fast writes.

When you switch to, say, 8 drives, the numbers start to change a bit.
RAID5 = 7TB available, 1 lost.
RAID6 = 6TB available, 2 lost.
RAID10 = 4TB available, 4 lost.