SATA Raid (stress test..)

[Nikolas Britton]

On 3/5/06, Beastie <beastie at mra.co.id> wrote:
> Nikolas Britton wrote:
> On 3/3/06, Alex Zbyslaw <xfb52 at dial.pipex.com> wrote:

> Nikolas Britton wrote:


>
> Please can you be careful when you attribute your comments. You've sent
this
> email "to" me, and left only my name in the attributions as if I
were
> someone suggesting either dd or diskinfo as accurate benchmarks,
when in
> fact my contribution was to suggest unixbench and sandra-lite.
Maybe you
> hate those too, in which case you can quote what I said
in-context and
> rubbish that at your pleasure.



> Yes I see your point, it does look like I'm replying to something you
wrote.
> This was a oversight and I am sorry.



> OK.


> Remember that 105MB/s number I quoted above?, that's just the
sustained read
> transfer rate for a big ass file, I don't need to work
with big ass files. I
> need to work with 15MB files (+/- 5MB). After
buying the right disks,
> controller, mainboard etc. and lots of tuning
with the help of iozone I get:
> 200 - 350MB/s overall (read, write,
etc.) for files less then or equal to
> 64MB*.

So anyways, that's what iozone can do for you. google it and
> you'll
find out more stuff about it.



> Thanks for the info. I think I can only dream about numbers like like
yours.
> Iozone looks to be in the ports so I see some of my weekend
disappearing
> looking at it :-)


>
> It runs on over two dozen operating systems, including windows. Their are
> two primary reasons I can get such high transfer rates from simple SATA
> drives. The first one was the selection of the mainboard that had a PCI-X
> slots, I built this system before PCI-Express mainboards and controllers hit
> the market. The PCI bus is severely restricted and obsolete, I'm simply
> going to post the theoretical maximum throughput in MB/s for the various bus
> standards: f(x,y) = x-bits * y-MHz / 8 = maximum theoretical throughput in
> MB/s PCI: 32 bits * 33 Mhz / 8 = 132 MB/s (standard PCI bus found on every
> pc) PCI: (32bits, 66MHz) = 264MB/s (Cards are commonplace, mainboards
> aren't) PCI-X: (64, 33) = 264MB/s (obsolete, won't find it on new boards.)
> PCI-X: (64, 66) = 528MB/s (Commonplace.) PCI-X: (64, 100) = 800 PCI-X: (64,
> 133) = 1064 (Commonplace.) PCI-X: (64, 266) = 2128 PCI-X: (64, 533) = 4264
> (very hard to find, even on high-end equipment.) PCI-X version 1 (66MHz -
> 133MHz) and PCI-X version 2 (266MHz - 533MHz). PCI-X is backwards compatible
> with PCI and slower versions of PCI-X, for example you can put a standard
> PCI card in a PCI-X 533MHz slot and it will simply run at (32, 33) similarly
> a 66 MHz PCI card will run at (32, 66) and so on and so forth. PCI-X is also
> forwards compatible in the fact that you can run a 133MHz PCI-X card in a
> standard (32, 33) PCI slot. Because of the backwards and an forwards
> compatibly I feel that PCI-X is superior to PCI-Express, *BUT* PCI-Express
> moving forwards is far far superior to PCI & PCI-X because it does not have
> 13 years of legacy to remain compatible with, it's cheaper to produce, and
> it's already in lower-end desktop systems as a replacement for AGP thanks to
> all the gamers. A few years from now PCI will end up where ISA / EISA are.
> I'm veering way off topic so I will not go into anymore details about PCI,
> PCI-X, and PCI-Express. Google around for the shortcomings of PCI / PCI-X
> and why PCI-Express is the future. PCI-Express: PCIe is not compatible with
> PCI or PCI-X (except for PCIe to PCI bridging) and it's just, well, totally
> different from the PCI spec and I'm already way off topic so again just
> google the details. It's theoretical maximums are expressed in Gigabits per
> second but I will convert them to MB/s for comparison with PCI and PCI-X.
> x1: 2.5Gbps = 312.5MB/s x2: 625MB/s x4: 1250MB/s x8: 2500MB/s x12: 3750MB/s
> x16: 5000MB/s x32: 10000MB/s Anyways back on topic, what was the topic? Oh
> yes, why you won't see 200MB/s - 350MB/s if your using a standard PCI slot.
> If you look back up all the way at the top you will see that the standard
> PCI bus is a crap shoot and that it's limited to a theoretical maximum of
> 132 MB/s. What this means is that your RAID controller and the disks
> attached to it and the cache buffers attached to the disks are all capped at
> that theoretical maximum of 132MB/s. Then you have to take into account that
> the PCI bus is shared with other devices such as the network card, video
> card, USB, etc. Your RAID controller has to fight will all these devices and
> a 1Gbit NIC card can eat up 125MB/s (12.5MB/s for a 100Mbit NIC). The next
> reason for those high gains is because I picked drives with 16MB cache
> buffers and that I'm insane enough to run a production server with the
> write-back cache policy enabled on the array controller and enabling the
> write cache on the disks. This is stupidly insane unless you've planned for
> the worsts. The worst case scenario would be that you corrupt the array into
> an unrepairable state and loose everything if you had a power failure. --
> BSD Podcasts @ http://bsdtalk.blogspot.com/
> _______________________________________________
> freebsd-questions at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-questions
> To unsubscribe, send any mail to
> "freebsd-questions-unsubscribe at freebsd.org"
> attach iozone result of amrd0 with 4 spindle Seagate Baracuda 300 Gb SATA II
> (1 hotspare)
> w/ Intel SRCS16 PCI-X
> Is that fast or what ? :)
>

I'll have to take a closer look, but the first thing I noticed in your
test report is that you are only using a 1MB test file. You should run
a test that will also max out the on disk / controller buffers. I
think the Baracuda's have a 16MB buffers (16MBx4=64MB) so try a 128MB
test file. Also be nice to see more detailed hardware specs about the
system and what version of FreeBSD are you running.

Thanks.

--
BSD Podcasts @ http://bsdtalk.blogspot.com/