suggestions for SATA RAID cards

[Gibson, Jasen (GE Indust, ConsInd, consultant)]

-----Original Message-----
Date: Mon, 2 Oct 2006 18:02:30 +0100 (BST)
From: Chris Hedley <cbh-freebsd-current at groups.chrishedley.com>
Subject: Re: suggestions for SATA RAID cards

On Tue, 29 Aug 2006, Barkley Vowk wrote:
>> Little late to the game on this post...

>Even later here, but I thought I may as well throw in my tuppence worth 
>since I'm going through this again at the moment.
>
>I have an Adaptec 2410SA coupled to 4 250GB discs.  I strongly recommend 
>avoiding it: terrible performance, on RAID5 or 10 partitions I get around 
>30-40 MB/s on reads and between 3 and 6 MB/s on writes.  Performance is 
>probably better if using write back caching on the card and discs, but 
>nobody here will be doing that, right?  Its other main "issue" is more 
>timeouts than you can shake a stick at under load, and unless you can kill 
>off the offending process, the reset button's the only way out.  I tested 
>its ability to rebuild its sets after simulating a disc failure by 
>removing one of the hot-swap trays, which it repaired without any issues 
>other than the time it took; I eventually got fed up with hanging around 
>for it to finish, but it was somewhere between 6 and 12 hours.
>
>I suspect I'll go for the Areca this time around, but I think I'll 
>approach it with rather more caution as I don't want to find myself with 
>another lemon like the 2410SA!
>
>Chris.

I'm using the same card, Chris.  My setup is a Tyan S2881 dual Opteron board and 4 WD RE2 400GB drives in a RAID 5.
My speed varies greatly, depending how large a file I'm accessing (which is an expected result, I think).  Files less than a GB, write at about around 130MB/s, read between 400-600MB/s  (have not ruled out whether cache is helping boost this number).  Files over 10GB this drops to a read speed of 40GB, like you've seen.  But the write speed will stay up at around 90!  It's very odd.  I've been seeing similar posts to this for a while now, concerning Highpoint, 3ware, Qlogic, and Adaptec cards and freebsd 5.4+; it almost makes me wonder if there's not another factor here, drivers or something.
I have not seen any of the timeouts you mention.  Considering I'm running a Mysql-driven forum off of this, Mysql should be hanging up quite often if this were happening.
I've just been using dd to come up with these numbers.  Are there any true disk benchmarking utils out there for BSD?