(S)ATA performance in FBSD 6.2/7.0

[Eric Anderson]

On 03/02/07 09:28, Brooks Davis wrote:
> On Fri, Mar 02, 2007 at 10:38:35AM +0100, O. Hartmann wrote:
>> The last days I tried to figure out why some of my lab's FreeBSD boxes 
>> and also mine at home seem to be outperformed by some Linux setups 
>> around here and I saw something interesting.
>>
>> On my lab's FreeBSD 6.2/i386 box (ASUS P4P800, ICH5 with two SATA 150 
>> ports, two SATA 300 drives attached) I copied big files (~ 5GB) from one 
>> drive to another while the box didn't do anything else than copying. I 
>> watched the copy process via 'systat -vmstat 1' and realized, that the 
>> value of 'KB/t' never go byond 128 (128kb buffer limit?). But more 
>> frustrating, I never got beyond 33 MB/s transfer rate although 
>> bonni/bonni++ told me both drives are capable doing much more (~75 MB/s 
>> each).
>> At home, I use a FreeBSD 7.0-CURRENT box on an ASUS 
>> A8N32-SLI/nForce4-SLI based box, amd64 (no 32Bit compatibility). Two 
>> Hitachi T7K250 250 GB/SATA II drives build up a RAID 0 (nVidia 
>> MediaShield), and additionally there is a SAMSUNG Spinpoitn SP2004C 
>> attached to the controller. bonni results in 55 MB/s for the SP2004C 
>> alone and gives ~ 65 - 70 MB/s for the Hitachis, each and roughly 115 
>> MB/s for the RAID 0. But copying from the single drive to the RAID 0 or 
>> from the RAID 0 to the single drive also reaches this oscure 33 MB/s 
>> boundary!
>>
>> In the first place I thought the older i386 hardware has some 
>> hard-limits, but we have several boxes of the exact same hardware around 
>> here and a wide spread Linux and Windows utilization and on those boxes 
>>  equipted with more than one harddrive (PATA or SATA) the effective 
>> transfer rate shown up is about 50 - 65 MB/s as expected with copying a 
>> big 5G file from one drive to another.
>>
>> The hardwrae limit is completely nonsense when it comes to the AMD64 box 
>> with newer hardware.
>>
>> Before digging into this problem deeper with benchmarks, could anyone 
>> explain why FreeBSD reaches this 33 MB/s limit (sounds like UDMA 33 
>> defaults, but on both boxes nForce4 and ICH5 controller are recognized 
>> and show up with SATA300 or SATA150 capabilities, respective)? May I 
>> have some knobs I'm not aware of to tune disk performance?
>>
>> I would appreciate any coments on that and if someone has some good 
>> ideas how to benchmark those subjects, please let me know.
> 
> One thing to keep in mind is that it matters a lot were on the disk you
> place the data due to the higher angular density of data at the outside
> of the disk.  The results you are seeing are close to consistant with
> the kind of results I'd expect to see from writing at opposiste edges
> of the disk.  The 33MB/s is suspious ane may diserve investigation, but
> make sure you are writing to the same part of the disk if you want to
> compare disk IO rates.
> 
> There's an example of IO rates on recent large SATA disks:
> 
> http://storagereview.com/articles/200607/500_2.html
> 
> Also, you should time the actual copy and do the math to verify that
> vmstat is actually producing valid results.  It's possible there's a bug
> in vmstat or the underlying statistics it uses.



I usually use gstat instead, but it might also be off (although my tests 
  in the past have not proven that).

Eric