ZFS obn FreeBSD hardware model for 48 or 96 sata3 paths...

[Peter Jeremy]

On 2011-Sep-16 18:22:06 -0700, Jason Usher <jusher71 at yahoo.com> wrote:
>1) immediately support 48 internal sata3 drives at full bandwidth -
>every drive has independent path to CPU

This would seem to be overkill - no current HDD can saturate a SATA3
channel.  And I suspect you will run into DRAM bandwidth issues well
before you saturate 48 SATA3 channels.

>Next, I see a lot of implementations done with LSI adaptors - is this
>as simple as choosing (3) LSI SAS 9201-16i for the 48 internal drives
>and (3) LSI SAS 9201-16e for the external drives ?

I can't comment on driver support but I'd start by checking for
motherboards that have 6 PCIe x16 slots with all lanes available.

>I would also like to spec and use a ZIL+L2ARC and am not sure where
>to go ... the system will be VERY write-biased and use a LOT of
>inodes - so lots of scanning of large dirs with lots of inodes and
>writing data.  Something like 400 million inodes on a filesystem with
>an average file size of 150 KB.

ZIL will only be useful if you do lots of sync writes.  L2ARC won't
help write performance.  Heavy write load implies you want mirroring
rather than RAIDZ and mirroring 60TB with 48 spindles means 3TB disks.

>- can I just skip the l2arc and just add more RAM ?

Definitely - I'd be looking at around 200GB RAM - and you might need
to tweak the ZFS (particularly) ARC parameters to suit the workload.

>- provided I maintain the free pcie slot(s) and/or free 2.5" drive
>slots, can I always just add a ZIL after the fact ?

Yes.

-- 
Peter Jeremy
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 196 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-fs/attachments/20110919/da8fce9e/attachment.pgp