(S)ATA performance in FBSD 6.2/7.0

[Brooks Davis]

On Fri, Mar 02, 2007 at 10:38:35AM +0100, O. Hartmann wrote:
> The last days I tried to figure out why some of my lab's FreeBSD boxes 
> and also mine at home seem to be outperformed by some Linux setups 
> around here and I saw something interesting.
> 
> On my lab's FreeBSD 6.2/i386 box (ASUS P4P800, ICH5 with two SATA 150 
> ports, two SATA 300 drives attached) I copied big files (~ 5GB) from one 
> drive to another while the box didn't do anything else than copying. I 
> watched the copy process via 'systat -vmstat 1' and realized, that the 
> value of 'KB/t' never go byond 128 (128kb buffer limit?). But more 
> frustrating, I never got beyond 33 MB/s transfer rate although 
> bonni/bonni++ told me both drives are capable doing much more (~75 MB/s 
> each).
> At home, I use a FreeBSD 7.0-CURRENT box on an ASUS 
> A8N32-SLI/nForce4-SLI based box, amd64 (no 32Bit compatibility). Two 
> Hitachi T7K250 250 GB/SATA II drives build up a RAID 0 (nVidia 
> MediaShield), and additionally there is a SAMSUNG Spinpoitn SP2004C 
> attached to the controller. bonni results in 55 MB/s for the SP2004C 
> alone and gives ~ 65 - 70 MB/s for the Hitachis, each and roughly 115 
> MB/s for the RAID 0. But copying from the single drive to the RAID 0 or 
> from the RAID 0 to the single drive also reaches this oscure 33 MB/s 
> boundary!
> 
> In the first place I thought the older i386 hardware has some 
> hard-limits, but we have several boxes of the exact same hardware around 
> here and a wide spread Linux and Windows utilization and on those boxes 
>  equipted with more than one harddrive (PATA or SATA) the effective 
> transfer rate shown up is about 50 - 65 MB/s as expected with copying a 
> big 5G file from one drive to another.
> 
> The hardwrae limit is completely nonsense when it comes to the AMD64 box 
> with newer hardware.
> 
> Before digging into this problem deeper with benchmarks, could anyone 
> explain why FreeBSD reaches this 33 MB/s limit (sounds like UDMA 33 
> defaults, but on both boxes nForce4 and ICH5 controller are recognized 
> and show up with SATA300 or SATA150 capabilities, respective)? May I 
> have some knobs I'm not aware of to tune disk performance?
> 
> I would appreciate any coments on that and if someone has some good 
> ideas how to benchmark those subjects, please let me know.

One thing to keep in mind is that it matters a lot were on the disk you
place the data due to the higher angular density of data at the outside
of the disk.  The results you are seeing are close to consistant with
the kind of results I'd expect to see from writing at opposiste edges
of the disk.  The 33MB/s is suspious ane may diserve investigation, but
make sure you are writing to the same part of the disk if you want to
compare disk IO rates.

There's an example of IO rates on recent large SATA disks:

http://storagereview.com/articles/200607/500_2.html

Also, you should time the actual copy and do the math to verify that
vmstat is actually producing valid results.  It's possible there's a bug
in vmstat or the underlying statistics it uses.

-- Brooks
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-questions/attachments/20070302/774144a7/attachment.pgp