Very low disk performance on 5.x

[Allen]

At 09:28 5/2/2005, Steven Hartland wrote:
>----- Original Message ----- From: "Poul-Henning Kamp" <phk at phk.freebsd.dk>
>>>Wouldn't this be a problem for writes then too?
>>I presume you would only compare read to write performance on a RAID5
>>device which has battery backed cache.
>>Without a battery backed cache (or pretending to have one) RAID5
>>write performance is abysmall no matter which alignment you use.
>
>This not what's been reported we are seeing writes that are 2x the
>speed of reads. Give the additional overhead that writes encure
>on RAID5 this should never be the case. The results I go in my tests
>where:

FWIW I have exactly this situation on a Mylex controller I have, if I 
enable the write cache on the card.  For some reason or another, doing so 
results in marginally improved write speeds, but sustained read speeds that 
drop well below 1/4 of the values they get when the *write* cache is disabled.

I have never figured out nor discovered a satisfactory explaination for why 
this is the case, and unlike Erics situation, this is consistent for me on 
that card across all OSes and tests.  May be some more fuel for the fire 
however, to someone in the know.

Also you should keep in mind, there could simply be some really goofy 
controller option enabled, that forces the RAID5 to behave in a "degraded" 
state for reads -- forcing it to read up all the other disks in the stripe 
and calculate the XOR again, to make sure the data it read off the disk 
matches the checksum.  It's rare, but I've seen it before, and it will 
cause exactly this sort of RAID5 performance inversion.  Since the XOR is 
recalculated on every write and requires only reading up one sector on a 
different disk, options that do the above will result in read scores 
drastically lower than writes to the same array.