PCIe SATA HBA for ZFS on -STABLE

[Alexander Motin]

On 07.06.2011 05:33, Matthew Dillon wrote:
>      The absolute cheapest solution is to buy a Sil-3132 PCIe card
>      (providing 2 E-SATA ports), and then connect an external port multiplier
>      to each port.  External port multiplier enclosures typically support
>      5 drives each so that would give you your 10 drives.
>
>      Even the 3132 is a piss-ant little card it does support FIS-Based
>      switching so performance will be very good... it will just be limited
>      to SATA-II speeds is all.

SiI3132 is indeed good for it's price and it is quite good for random 
I/O. But at burst speeds it is limited lower then SATA-II. Even lower 
then PCIe 1.0 x1 it uses. IIRC I've seen about 150MB/s from one port and 
about 170MB/s from two.

If burst rate is important, SiI3124 chip is much better -- up to about 
900MB/s measured from 4 ports. The only issue is PCI-X interface: either 
motherboard with PCI-X needed, or card with PCIe x8 bridge (like these 
http://www.addonics.com/products/host_controller/adsa3gpx8-4e.asp), but 
last case is too expensive.

There are also much cheaper (~$50) PCIe x1 bridge SiI3124 cards 
(http://www.sybausa.com/productInfo.php?iid=537). They are not so fast 
-- about 200MB/s, but still more then SiI3132. And they still have 4 
SATA ports.

>      For SSDs you want to directly connect the SSD to a mobo SATA port and
>      then either mount the SSD in the case or mount it in a hot-swap gadget
>      that you can screw into a PCI slot (it doesn't actually use the PCI
>      connector, just the slot).  A SATA-III port with a SATA-III SSD really
>      shines here and 400-500 MBytes/sec random read performance from a single
>      SSD is possible, but it isn't an absolute requirement.  A SATA-II port
>      will still work fine as long as you don't mind maxing out the bandwidth
>      at 250 MBytes/sec.

Agree. Intel on-board ports rock! Recently I've built new system with 
two OCZ Vertex 3 SSDs connected to 6Gbps SATA ports on Intel Sandy 
Bridge class motherboard. UFS on top of graid RAID0 volume gives me 
about 950MB/s on both read and write!

>      To get robust hot-swap enclosures you either need to go with SAS or you
>      need to go with discrete SATA ports (no port multiplication), and the
>      ports have to support hot-swap.  The best hot-swap support for an AHCI
>      port is if the AHCI chipset supports cold-presence-detect (CPD), and
>      again Mobo AHCI chipsets usually don't.  Hot-swap is a bit hit or miss
>      without CPD because power savings modes can effectively prevent hot-swap
>      detect from working properly.  Drive disconnects will always be detected
>      but drive connects might not be.

I would say it depends. In some cases it is easier to detect hot-plug 
then hot-unplug, as device sends COMINIT that should wake up port even 
from power-save state. With ICH10, for example, I've managed to make 
both hot plug and unplug work even with power-management enabled: 
hot-plug via tracking COMINIT, unplug via it's CPD capability. Without 
PM it "just works". :)

-- 
Alexander Motin