8.0-RELEASE/amd64 - full ZFS install - low read and write disk performance

[Thomas Burgess]

It depends on the bandwidth of the bus that it is on and the controller
itself.

I like to use pci-x with aoc-sat2-mv8 cards or pci-e cards....that way you
get a lot more bandwidth..

On Mon, Jan 25, 2010 at 3:32 AM, Dan Naumov <dan.naumov at gmail.com> wrote:

> On Mon, Jan 25, 2010 at 9:34 AM, Dan Naumov <dan.naumov at gmail.com> wrote:
> > On Mon, Jan 25, 2010 at 7:33 AM, Bob Friesenhahn
> > <bfriesen at simple.dallas.tx.us> wrote:
> >> On Mon, 25 Jan 2010, Dan Naumov wrote:
> >>>
> >>> I've checked with the manufacturer and it seems that the Sil3124 in
> >>> this NAS is indeed a PCI card. More info on the card in question is
> >>> available at
> >>> http://green-pcs.co.uk/2009/01/28/tranquil-bbs2-those-pci-cards/
> >>> I have the card described later on the page, the one with 4 SATA ports
> >>> and no eSATA. Alright, so it being PCI is probably a bottleneck in
> >>> some ways, but that still doesn't explain the performance THAT bad,
> >>> considering that same hardware, same disks, same disk controller push
> >>> over 65mb/s in both reads and writes in Win2008. And agian, I am
> >>> pretty sure that I've had "close to expected" results when I was
> >>
> >> The slow PCI bus and this card look like the bottleneck to me. Remember
> that
> >> your Win2008 tests were with just one disk, your zfs performance with
> just
> >> one disk was similar to Win2008, and your zfs performance with a mirror
> was
> >> just under 1/2 that.
> >>
> >> I don't think that your performance results are necessarily out of line
> for
> >> the hardware you are using.
> >>
> >> On an old Sun SPARC workstation with retrofitted 15K RPM drives on
> Ultra-160
> >> SCSI channel, I see a zfs mirror write performance of 67,317KB/second
> and a
> >> read performance of 124,347KB/second.  The drives themselves are capable
> of
> >> 100MB/second range performance. Similar to yourself, I see 1/2 the write
> >> performance due to bandwidth limitations.
> >>
> >> Bob
> >
> > There is lots of very sweet irony in my particular situiation.
> > Initially I was planning to use a single X25-M 80gb SSD in the
> > motherboard sata port for the actual OS installation as well as to
> > dedicate 50gb of it to a become a designaed L2ARC vdev for my ZFS
> > mirrors. The SSD attached to the motherboard port would be recognized
> > only as a SATA150 device for some reason, but I was still seeing
> > 150mb/s throughput and sub 0.1 ms latencies on that disk simply
> > because of how crazy good the X25-M's are. However I ended up having
> > very bad issues with the Icydock 2,5" to 3,5" converter jacket I was
> > using to keep/fit the SSD in the system and it would randomly drop
> > write IO on heavy load due to bad connectors. Having finally figured
> > out the cause of my OS installations to the SSD going belly up during
> > applying updates, I decided to move the SSD to my desktop and use it
> > there instead, additionally thinking that my perhaps my idea of the
> > SSD was crazy overkill for what I need the system to do. Ironically
> > now that I am seeing how horrible the performance is when I am
> > operating on the mirror through this PCI card, I realize that
> > actually, my idea was pretty bloody brilliant, I just didn't really
> > know why at the time.
> >
> > An L2ARC device on the motherboard port would really help me with
> > random read IO, but to work around the utterly poor write performance,
> > I would also need a dedicaled SLOG ZIL device. The catch is that while
> > L2ARC devices and be removed from the pool at will (should the device
> > up and die all of a sudden), the dedicated ZILs cannot and currently a
> > "missing" ZIL device will render the pool it's included in be unable
> > to import and become inaccessible. There is some work happening in
> > Solaris to implement removing SLOGs from a pool, but that work hasn't
> > yet found it's way in FreeBSD yet.
> >
> >
> > - Sincerely,
> > Dan Naumov
>
> OK final question: if/when I go about adding more disks to the system
> and want redundancy, am I right in thinking that: ZFS pool of
> disk1+disk2 mirror + disk3+disk4 mirror (a la RAID10) would completely
> murder my write and read performance even way below the current 28mb/s
> / 50mb/s I am seeing with 2 disks on that PCI controller and that in
> order to have the least negative impact, I should simply have 2
> independent mirrors in 2 independent pools (with the 5th disk slot in
> the NAS given to a non-redundant single disk running off the one
> available SATA port on the motherboard)?
>
> - Sincerely,
> Dan Naumov
> _______________________________________________
> freebsd-fs at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-fs
> To unsubscribe, send any mail to "freebsd-fs-unsubscribe at freebsd.org"
>