RHEL to FreeBSD file server

[Zaphod Beeblebrox]

On Fri, Nov 16, 2012 at 3:17 AM, Rainer Duffner <rainer at ultra-secure.de> wrote:
> Am Fri, 16 Nov 2012 01:03:19 -0500
> schrieb Zaphod Beeblebrox <zbeeble at gmail.com>:

>> The only price I see is $2449 on Dell's website.  Ouch.  For 8G.

> I saw it for 1800€ somewhere.
> FusionIO is 10k for 320G.
>
> But I've never seen FusionIO used in a Fileserver - probably doesn't
> make sense, price-wise.
>
> IIRC, the OP wanted to build a 16T fileserver. So he will have to spend
> some cash anyway.  I think it's a safe assumption that Nexenta and its
> partners have spent quite some time evaluating the various options.

That seems about the same price.  I've stamped out a few of the
following builds which are surprisingly cheap and fast.

You start with an ICH10 motherboard where the 6 ICH10 SATA ports
support port multipliers (this is random and you may have to test ---
some do and some don't).  Bonus marks if your motherboard also has a
few other SATA ports.  Check out "gamer" boards from the ASUS "RoG"
series.

Now... port multipliers are built into quite a few things ... 4 drive
external boxes are fairly easy to find, but if you look, the port
multiplier itself (1-to-4 or 1-to-5) is available separately for about
$40.  attach port multipliers to each of the 6 ICH10 ports and go on
to connect 24 (or 30) drives.  If you're using "green" 2T's, the speed
of 4 disks on one channel is about half of the speed of
1-disk-per-channel.

If your motherboard has 2 extra SATA (sometimes 6G, for instance),
they likely _don't_ support multipliers (this seems deliberate).  But
you can have a 2 disk raid-1 boot of either FFS or ZFS.  Another
choice is USB for either flash or HD boot.

Put as much memory as it supports.  Desktop board will often take 8G
or 16G now... or more.  You'll want a big power supply and an nice i5
:).

You'll get close to wire speed GigE out of this setup.  3x8 vdevs in
RAID-Z1 or Z2 work nicely.

The reason I say all this... is that this config runs about $3500-ish
here in Canada (where green 2T's are ~$109).  The ZeusRAM drive up
there is 2/3 of that.