PCIe SATA HBA for ZFS on -STABLE
dillon at apollo.backplane.com
Tue Jun 7 04:33:07 UTC 2011
:I'm not on the -STABLE list so please reply to me.
:I'm using an Intel Core i3-530 on a Gigabyte H55M-D2H motherboard with 8 x
:2TB drives & 2 x 1TB drives.
:The plan is to have the 1 TB drives in a zmirror and the 8 in a raidz2.
:Now the Intel chipset has only 6 on board SATA II ports so ideally I'm
:looking for a non RAID SATA II HBA to give me 6 extra ports (4 min).
:Why 6 extra ?
:Well the case I'm using has 2 x eSATA ports so 6 would be ideal, 5 OK, and 4
:the minimum I need to do the job.
:What do people recommend for 8-STABLE as a PCIe SATA II HBA for someone
:using ZFS ?
:Not wanting to break the bank.
:Not interested in SATA III 6GB at this time... though it could be useful if
:I add an SSD for... (is it ZIL ?).
:Can this be added at any time ?
:The main issue is I need at least 10 ports total for all existing drives...
:ZIL would require 11 so ideally we are talking a 6 port HBA.
The absolute cheapest solution is to buy a Sil-3132 PCIe card
(providing 2 E-SATA ports), and then connect an external port multiplier
to each port. External port multiplier enclosures typically support
5 drives each so that would give you your 10 drives.
Even the 3132 is a piss-ant little card it does support FIS-Based
switching so performance will be very good... it will just be limited
to SATA-II speeds is all.
Motherboard AHCI-based SATA ports typically do NOT have FIS-Based
switching support (this would be the FBSS capability flag when the AHCI
driver probes the chipset). This means that while you can attach an
external port multiplier enclosure to mobo SATA ports (see later
on E-SATA vs SATA), read performance from multiple drives concurrently
will be horrible. Write performance will still be decent due to drive
write caches despite being serialized.
On E-SATA vs SATA. Essentially there are only two differences between
E-SATA and SATA. One is the cable and connector format. The other is
hot swap detection. Most mobo SATA ports can be strung out to E-SATA
with an appropriate adapter. High-end Intel ASUS mobos often come with
such adapters (this is why they usually don't sport an actual E-Sata
port on the backplane) and the BIOS has setup features to specify E-SATA
on a port-by-port basis.
For SSDs you want to directly connect the SSD to a mobo SATA port and
then either mount the SSD in the case or mount it in a hot-swap gadget
that you can screw into a PCI slot (it doesn't actually use the PCI
connector, just the slot). A SATA-III port with a SATA-III SSD really
shines here and 400-500 MBytes/sec random read performance from a single
SSD is possible, but it isn't an absolute requirement. A SATA-II port
will still work fine as long as you don't mind maxing out the bandwidth
at 250 MBytes/sec.
I can't help with any of the other questions. Someone also suggested
the MPS driver for FreeBSD, with caveats.
I'll add a caveat on the port multiplier enclosures. Nearly all such
enclosures use another SIL chipset internally and it works pretty well
EXCEPT that it isn't 100% dependable if you try to hot-swap drives in the
enclosure while other drives in the enclosure are active. So with that
caveat, I recommend the port multiplier enclosure as the cheapest solution.
To get robust hot-swap enclosures you either need to go with SAS or you
need to go with discrete SATA ports (no port multiplication), and the
ports have to support hot-swap. The best hot-swap support for an AHCI
port is if the AHCI chipset supports cold-presence-detect (CPD), and
again Mobo AHCI chipsets usually don't. Hot-swap is a bit hit or miss
without CPD because power savings modes can effectively prevent hot-swap
detect from working properly. Drive disconnects will always be detected
but drive connects might not be.
And even with discrete SATA ports the AHCI firmware on mobos does not
necessarily handle hot-swap properly. For example my Intel-I7 ASUS mobo
will generate spurious interrupts and status on a DIFFERENT discrete
SATA port when I hot swap on some other discrete SATA port, in addition
to generating the status interrupt on the correct port. So then it comes
down to the driver in the operating system properly handling the
spurious status and properly stopping and restarting pending commands
when necessary. So, again, it is best for the machine to be idle before
attempting a hot-swap.
Lots of caveats. Sorry... you can blame Intel for all the blasted issues
with AHCI and SATA. Intel didn't produce a very good chipset spec and
vendors took all sorts of liberties.
<dillon at backplane.com>
More information about the freebsd-stable