Optimal File System config for 2.5TB RAID5

[Danny Do]

Thanks for the concrete example of the pitfall of hardware RAID Jeremy.

I never had any problem with hardware driver, that's why I never thought of
it. But you are quite right! I should be avoiding hardware RAID whenever
possible. I get much more support and quicker response here than from
hardware vendor.

Ok, I have to pickup gVinum where I left it 4 years ago. Hopefully, the
software is stable now. 


Thanks again Jeremy Chadwick, Wojciech Puchar and this wonderful community,

Danny



-----Original Message-----
From: owner-freebsd-questions at freebsd.org
[mailto:owner-freebsd-questions at freebsd.org] On Behalf Of Jeremy Chadwick
Sent: Wednesday, 1 October 2008 5:45 AM
To: Danny Do
Cc: 'Wojciech Puchar'; freebsd-questions at freebsd.org
Subject: Re: Optimal File System config for 2.5TB RAID5

On Wed, Oct 01, 2008 at 04:49:27AM +0700, Danny Do wrote:
> I got Perc 4E-DI Embedded Raid Adapter (256MB) from DELL for my current
SCSI
> system. They said it's the enterprise class. I don't know much about the
> performance between software RAID and hardware RAID.

I'm not familiar with PERC (LSI) controllers, just for the record.

> Could you please tell me if this type of hardware RAID controller could
> match the software RAID you were talking about? 

What you're asking for is "too much" -- and this conversation is
starting to delve into freebsd-hardware, not freebsd-questions.

Unless someone out there has done full benchmarks comparing FreeBSD ZFS
or FreeBSD gvinum to a PERC 4E-DI, with all kinds of test cases (what
sort of server it is, what it's doing disk-wise, etc.), I doubt you'll
be able to get a conclusive answer here.  Such benchmarking would
require weeks of effort by someone.

Heck, I'm not even sure FreeBSD supports the PERC 4E-DI.

That said, if you go with that controller, you should be aware of the
following things: there are many problems with hardware RAID.

1) If the controller goes bad after the lifetime of the controller has
expired, there is very little chance the vendor will give you a
replacement controller that understands the metadata of the previous/bad
controller.

You are flat out stuck with that model of controller for the rest of
your life, unless the vendor can *guarantee* backwards compatibility
when providing a newer controller.  And I'm willing to bet money that
general technical support has no idea what "metadata" is, or any
technical details; they just know what they're told ("controller X is no
longer available, give them controller Y")

2) Driver support is often "iffy" with such controllers, at least under
FreeBSD.  FreeBSD SCSI CAM is quite reliable, so that's not the problem.
Here's some past evidence of mfi(4) and mpt(4) having problems
administrating arrays, or experiencing horrible performance, requiring
tuning be done and much troubleshooting:

http://wiki.freebsd.org/JeremyChadwick/Commonly_reported_issues

3) You are at the whim of the hardware RAID controller's BIOS.
Performance can be affected by bugs in the BIOS, or BIOS bugs can cause
you trouble down the road.  You have to ask yourself how much you
ultimately trust the technical support people at Dell vs. the FreeBSD
community.

4) Driver regressions may hurt you.  There may be a day when you go
to upgrade to FreeBSD 8.0 (when it becomes stable), only to find that
your controller isn't recognised, or has odd problems.  (I myself
just ran into this situation with -CURRENT last week, where my SATA
controller isn't detected, while works perfectly in RELENG_7).  You're
then "stuck" on an older FreeBSD until those problems can be worked
out.


The only hardware RAID controller I've seen praise for, under FreeBSD,
are Areca controllers.  I'm told the performance (on a purely general
level) is "absolutely incredible/blazing fast".  I don't know what
those people are comparing against, though.

Be aware that many developers, including folks like Matt Dillon (of
DragonflyBSD) and Ade Lovett (very familiar with filers and disk
storage) recommend you *completely avoid* hardware RAID controllers or
on-motherboard RAID (e.g. Intel MatrixRAID), and go with OS-based RAID
(ZFS, gvinum, or standalone UFS2+SU filesystems).

If you reach a point where disk I/O on that server is becoming so
heavy that you feel you need a hardware RAID controller, that would
be when you should come back to the list (freebsd-stable,
freebsd-hardware, or freebsd-isp) to discuss the problems you're
having with performance.

-- 
| Jeremy Chadwick                                jdc at parodius.com |
| Parodius Networking                       http://www.parodius.com/ |
| UNIX Systems Administrator                  Mountain View, CA, USA |
| Making life hard for others since 1977.              PGP: 4BD6C0CB |

_______________________________________________
freebsd-questions at freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscribe at freebsd.org"