SAS5IR performance issue with Dell 860

[韓家標 Bill Hacker]

Espen Tagestad wrote:
> Tom Judge wrote:
>> Espen Tagestad wrote:
>>> We recently bought 3 new Dell 860 servers with the onboard SAS5/IR 
>>> SATA RAID-controller. They seem to be quite well spec'ed servers with 
>>> management and everything - but I am experiencing av major 
>>> performance issue with the disc i/o. On write I get at max 7-8MB/sec, 
>>> while read gives a bit more (11MB/sec). I tried first with 
>>> 6.2-RELEASE, and then upgraded to 6.3-PRERELEASE without any better 
>>> results.
>>
>> You will also need to set the hw.mpt.enable_sata_wc sysctl in 
>> loader.conf.
> 
> There isn't any hw.mpt.enable_sata_wc sysctl available on my systems. It 
> is 6.3-PRERELEASE, but as I recon there wasn't any *mpt* sysctls on the 
> latest 6.2-RELEASE either. Is it deprecated? Is there any other options? 
> I can see a hw.ata.wc but that one is set to 1 which I presume equals 
> enabled.
> 

It is not in the man page, and neither sysctl -a nor sysctl  -a -o shows it, 
even on 6.3-PRE.  Unless you have used it.

It should show up in sysctl -a -o only after first use.

> Anyway - the write cache, is that something that is set on the raid 
> controller, or is it a buffer in the FreeBSD kernel that takes care of 
> the caching?

Both and more.

The BSD buffers can generally be left alone to 'do the right thing'.

Buffers on the drive itself, likewise.

Mucking about with either is likely to do more harm than good overall unless 
your only goal is fast (looking) benchmarks. In which case attach an old Zip 
drive, remove the media, and watch truly astonishing benchmarks

... as the system does r/w to/from the in-RAM cache. Only.

;-)


Buffers 'managed by' the controller, whether physically onboard or an allocated 
chunk of system RAM depend on the needs of your application.

EX: A PostgreSQL DB is 'transactionally aware' and has its own Write Ahead 
Logging and commit/rollback tools. It is generally fast AND safe with 
Softupdates OFF for the mount-point or device where it stores its data and 
controller cache set to 'write through' rather than 'write back'.

A differently prioritized application might need either write-back caching and 
Softupdates. One, the other, neither, or both.  It is ever a tradeoff between 
recoverability and (apparent) speed, and too much tuning can be 
counterproductive for the more general cases.

I say 'apparent' because at the end of the day, the only writes that do not have 
to be made at all are those 'known to have' gone stale while still in the OS 
buffers. Or - better yet - while in L1/L2/L3 CPU cache...

;-)

> As the commit note you sent me said - to ensure absolutely 
> best data integrity the write cache should be left switched off. But 
> write performance of 7-8MB/sec is just too low for that - is the 
> controller /sata drives really that slow?
> 

Depends very much on what (else) they and their drives are doing. Drive head 
positioning is the killer that channel speed can't do much about.

If the controller and drives are dedicated solely to one app or file transfer, 
do no logging, have no system tools or such on the same platter(s) that might 
need interruptions to their measured task, are able to do r/w in 
damn-near-slew-mode, they should be much faster.

If there are any system-relative mounts on the same spindle(s) (/var/log comes 
first to mind) - then probably not. Heads have to go elsewhere and back.

Real-world *NIX boxen are rarely in the position of concentrating on just one 
I/O task at a time.

...and you still haven't said what you are using to derive the numbers, what 
sort of drives are at the end of the cable, or if this is an application or test 
suite, and/or if the r/w is to drives that do NOT do anything else......

??

Bill