ZFS or UFS for 4TB hardware RAID6?

[Maxim Khitrov]

On Mon, Jul 13, 2009 at 4:08 PM, Richard Mahlerwein<mahlerrd at yahoo.com> wrote:
>> > Just as a question: how ARE you planning on backing
>> this beast up?  While I don't want to sound like a
>> worry-wort, I have had odd things happen at the worst of
>> times.  RAID cards fail, power supplies let out the magic
>> smoke, users delete items they really want back... *sigh*
>>
>> Rsync over ssh to another server. Most of the data stored
>> will never
>> change after the first upload. A daily rsync run will
>> transfer one or
>> two gigs at the most. History is not required for the same
>> reason;
>> this is an append-only storage for the most part. A backup
>> for the
>> previous day is all that is required, but I will keep a
>> weekly backup
>> as well until I start running out of space.
>>
>> > A bit of reading shows that ZFS, if it's stable
>> enough, has some really great features that would be nice on
>> such a large pile o' drives.
>> >
>> > See http://wiki.freebsd.org/ZFSQuickStartGuide
>> >
>> > I guess the last question I'll ask (as any more may
>> uncover my ignorance) is if you need to use hardware RAID at
>> all?  It seems both UFS2 and ZFS can do software RAID
>> which seems to be quite reasonable with respect to
>> performance and in many ways seems to be more robust since
>> it is a bit more portable (no specialized hardware).
>>
>> I've thought about this one a lot. In my case, the hard
>> drives are in
>> a separate enclosure from the server and the two had to be
>> connected
>> via SAS cables. The 9690SA-8E card was the best choice I
>> could find
>> for accessing an external SAS enclosure with support for 8
>> drives.
>>
>> I could configure it in JBOD mode and then use software to
>> create a
>> RAID array. In fact, I will likely do this to compare
>> performance of a
>> hardware vs. software RAID5 solution. The ZFS RAID-Z option
>> does not
>> appeal to me, because the read performance does not benefit
>> from
>> additional drives, and I don't think RAID6 is available in
>> software.
>> For those reasons I'm leaning toward a hardware
>> implementation.
>>
>> If I go the hardware route, I'll try to purchase a backup
>> controller
>> in a year or two. :)
>>
>> > There are others who may respond with better
>> information on that front.  I've been a strong
>> proponent of hardware RAID, but have recently begun to
>> realize many of the reasons for that are only of limited
>> validity now.
>>
>> Agreed, and many simple RAID setups (0, 1, 10) will give
>> you much
>> better performance in software. In my case, I have to have
>> some piece
>> of hardware just to get to the drives, and I'm guessing
>> that hardware
>> RAID5/6 will be faster than the closest software
>> equivalent. Maybe my
>> tests will convince me otherwise.
>>
>> - Max
>
> I'd love to hear about any test results you may get comparing software with hardware raid.

I received the hardware yesterday. There was a last minute change due
to cost. Instead of getting 4x 2TB drives I opted for 6x 1TB. This
limits my future expansion a bit, but that may be a few years down the
line. On the plus side, I can get the 4TB of RAID6 that I originally
planned for and the performance should be better because of additional
disks in the array.

Sometime next week I'll install FreeBSD 8 and will then be able to run
a few benchmarks. After that I'll configure software RAID and repeat
the process. Are there any specific tests that you guys would like me
to run? Sequential read/write tests using dd are a given, beyond that
I'm not familiar with any ports under benchmarks/, so if you know of
anything good, tell me.

- Max