SATA Raid (stress test..)

Tue Mar 7 19:03:32 PST 2006

Nikolas Britton wrote:

>On 3/5/06, Beastie <beastie at mra.co.id> wrote:
>  
>
>>Nikolas Britton wrote:
>>On 3/3/06, Alex Zbyslaw <xfb52 at dial.pipex.com> wrote:
>>    
>>
>
>  
>
>>Nikolas Britton wrote:
>>    
>>
>
>
>  
>
>>Please can you be careful when you attribute your comments. You've sent
>>    
>>
>this
>  
>
>>email "to" me, and left only my name in the attributions as if I
>>    
>>
>were
>  
>
>>someone suggesting either dd or diskinfo as accurate benchmarks,
>>    
>>
>when in
>  
>
>>fact my contribution was to suggest unixbench and sandra-lite.
>>    
>>
>Maybe you
>  
>
>>hate those too, in which case you can quote what I said
>>    
>>
>in-context and
>  
>
>>rubbish that at your pleasure.
>>    
>>
>
>
>
>  
>
>>Yes I see your point, it does look like I'm replying to something you
>>    
>>
>wrote.
>  
>
>>This was a oversight and I am sorry.
>>    
>>
>
>
>
>  
>
>>OK.
>>    
>>
>
>
>  
>
>>Remember that 105MB/s number I quoted above?, that's just the
>>    
>>
>sustained read
>  
>
>>transfer rate for a big ass file, I don't need to work
>>    
>>
>with big ass files. I
>  
>
>>need to work with 15MB files (+/- 5MB). After
>>    
>>
>buying the right disks,
>  
>
>>controller, mainboard etc. and lots of tuning
>>    
>>
>with the help of iozone I get:
>  
>
>>200 - 350MB/s overall (read, write,
>>    
>>
>etc.) for files less then or equal to
>  
>
>>64MB*.
>>    
>>
>
>So anyways, that's what iozone can do for you. google it and
>  
>
>>you'll
>>    
>>
>find out more stuff about it.
>
>
>
>  
>
>>Thanks for the info. I think I can only dream about numbers like like
>>    
>>
>yours.
>  
>
>>Iozone looks to be in the ports so I see some of my weekend
>>    
>>
>disappearing
>  
>
>>looking at it :-)
>>    
>>
>
>
>  
>
>>It runs on over two dozen operating systems, including windows. Their are
>>two primary reasons I can get such high transfer rates from simple SATA
>>drives. The first one was the selection of the mainboard that had a PCI-X
>>slots, I built this system before PCI-Express mainboards and controllers hit
>>the market. The PCI bus is severely restricted and obsolete, I'm simply
>>going to post the theoretical maximum throughput in MB/s for the various bus
>>standards: f(x,y) = x-bits * y-MHz / 8 = maximum theoretical throughput in
>>MB/s PCI: 32 bits * 33 Mhz / 8 = 132 MB/s (standard PCI bus found on every
>>pc) PCI: (32bits, 66MHz) = 264MB/s (Cards are commonplace, mainboards
>>aren't) PCI-X: (64, 33) = 264MB/s (obsolete, won't find it on new boards.)
>>PCI-X: (64, 66) = 528MB/s (Commonplace.) PCI-X: (64, 100) = 800 PCI-X: (64,
>>133) = 1064 (Commonplace.) PCI-X: (64, 266) = 2128 PCI-X: (64, 533) = 4264
>>(very hard to find, even on high-end equipment.) PCI-X version 1 (66MHz -
>>133MHz) and PCI-X version 2 (266MHz - 533MHz). PCI-X is backwards compatible
>>with PCI and slower versions of PCI-X, for example you can put a standard
>>PCI card in a PCI-X 533MHz slot and it will simply run at (32, 33) similarly
>>a 66 MHz PCI card will run at (32, 66) and so on and so forth. PCI-X is also
>>forwards compatible in the fact that you can run a 133MHz PCI-X card in a
>>standard (32, 33) PCI slot. Because of the backwards and an forwards
>>compatibly I feel that PCI-X is superior to PCI-Express, *BUT* PCI-Express
>>moving forwards is far far superior to PCI & PCI-X because it does not have
>>13 years of legacy to remain compatible with, it's cheaper to produce, and
>>it's already in lower-end desktop systems as a replacement for AGP thanks to
>>all the gamers. A few years from now PCI will end up where ISA / EISA are.
>>I'm veering way off topic so I will not go into anymore details about PCI,
>>PCI-X, and PCI-Express. Google around for the shortcomings of PCI / PCI-X
>>and why PCI-Express is the future. PCI-Express: PCIe is not compatible with
>>PCI or PCI-X (except for PCIe to PCI bridging) and it's just, well, totally
>>different from the PCI spec and I'm already way off topic so again just
>>google the details. It's theoretical maximums are expressed in Gigabits per
>>second but I will convert them to MB/s for comparison with PCI and PCI-X.
>>x1: 2.5Gbps = 312.5MB/s x2: 625MB/s x4: 1250MB/s x8: 2500MB/s x12: 3750MB/s
>>x16: 5000MB/s x32: 10000MB/s Anyways back on topic, what was the topic? Oh
>>yes, why you won't see 200MB/s - 350MB/s if your using a standard PCI slot.
>>If you look back up all the way at the top you will see that the standard
>>PCI bus is a crap shoot and that it's limited to a theoretical maximum of
>>132 MB/s. What this means is that your RAID controller and the disks
>>attached to it and the cache buffers attached to the disks are all capped at
>>that theoretical maximum of 132MB/s. Then you have to take into account that
>>the PCI bus is shared with other devices such as the network card, video
>>card, USB, etc. Your RAID controller has to fight will all these devices and
>>a 1Gbit NIC card can eat up 125MB/s (12.5MB/s for a 100Mbit NIC). The next
>>reason for those high gains is because I picked drives with 16MB cache
>>buffers and that I'm insane enough to run a production server with the
>>write-back cache policy enabled on the array controller and enabling the
>>write cache on the disks. This is stupidly insane unless you've planned for
>>the worsts. The worst case scenario would be that you corrupt the array into
>>an unrepairable state and loose everything if you had a power failure. --
>>BSD Podcasts @ http://bsdtalk.blogspot.com/
>>_______________________________________________
>>freebsd-questions at freebsd.org mailing list
>>http://lists.freebsd.org/mailman/listinfo/freebsd-questions
>>To unsubscribe, send any mail to
>>"freebsd-questions-unsubscribe at freebsd.org"
>>attach iozone result of amrd0 with 4 spindle Seagate Baracuda 300 Gb SATA II
>>(1 hotspare)
>>w/ Intel SRCS16 PCI-X
>>Is that fast or what ? :)
>>
>>    
>>
>
>I'll have to take a closer look, but the first thing I noticed in your
>test report is that you are only using a 1MB test file. You should run
>a test that will also max out the on disk / controller buffers. I
>think the Baracuda's have a 16MB buffers (16MBx4=64MB) so try a 128MB
>test file. Also be nice to see more detailed hardware specs about the
>system and what version of FreeBSD are you running.
>
>Thanks.
>
>--
>BSD Podcasts @ http://bsdtalk.blogspot.com/
>
>  
>
 .

Second test with 128MB buffers (attach) on
- SATA II Seagate Baracuda
- PCI-X Intel SRCS16
- Intel Xeon 3.0 with 2 GB DDR RAM
- and Intel SE7320EP2 board
- FreeBSD-6.1 Pre-RELEASE

Thanks  before for good review and explanation. I need to be sure that 
there is no performance issue before i put this machine into production.

regards
reza