medium sized zfs nfs server (was: zpool degraded - 'UNAVAIL cannot open' functioning drive)

[Randy Bush]

[ i am a internet / research / software guy, and this pee cee hardware
  stuff scares the <bleep> outta me.  so i am seriously desperate with
  these questions and *highly* appreciative of the responses ]

>> [0] need to build a 20tb raidz2 system, kind of an nfs store to serve
>> some data collertor(s) and compute engine(s).  trying to sort out disk
>> controller and 10ge card decisions and the hardware / freebsd support
>> space is complex.
> 
> SuperMicro 24 disk chassis (CSE-846TQ-R900B)
> 24 1TB SATA disks
> (2) Areca 1231ML controllers with BBWC

thank you.  and the 10gige card?  thinking copper cx as it's just
between two or three hosts in the rack.

> Two RAID 5 partitions with no hot-spare disks will net you 19TB of
> formatted storage. If you ZFS stripe across the two RAID arrays, you'll
> get decent performance. If your data is compressible, enabling file
> system compression will easily store the 20TB you need.

actually, i was thinking what i hope is a bit more conservatively
(except for the last line)
  o exactly that chassis with a mobo with 8gb ram and > 2 cores
  o the goal is 20tb of spindles to get 10tb of data store
    [ may even wait some weeks for the 1.5tb drives, oink oink ]
  o two hot spare drives as this is in a remote rack
  o no hardware raid, have been burned by 3ware, ...
  o zfs's raidz2
  o (gasp!) -current for the zfs fixes

whack me harder with the clue bat, please

randy