Very low disk performance on 5.x

[Eric Anderson]

Arne Wörner wrote:
> --- Robert Watson <rwatson at FreeBSD.org> wrote:
> 
>>On Sat, 30 Apr 2005, Arne WXrner wrote:
>>
>>>3. The man page geom(4) of R5.3 says "The GEOM framework
>>>  provides an infrastructure in which "classes" can per-
>>>  form transformations on disk I/O requests on their path
>>>  from the upper kernel to the device drivers and back.
>>>
>>>Could it be, that geom slows something down (in some boxes the
>>>reading 
>>>ops are very slow; in my box the writing ops are very slow)?
>>
>>[...]
>>against a further offset region of ad0.  If they're against the
>>same bits of disk, the main difference here will be the
>>additional processing of the layers in the stack.  A little
>>bit of math is required to figure out the offset, but dd
>>should be usable to figure out the incremental  cost.
>>
> 
> I used the following commands:
> 1. dd if=/dev/ad0s2a bs=16k count=10000 of=/dev/null
> 2. dd if=/dev/ad0 iseek=2100357 bs=16k count=10000 of=/dev/null
> (I think I did the math correctly; indeed the read speed of my ad0
> varies between 45MB/sec (iseek=0) and 25MB/sec (in the end) with
> bs=128k).
> The results were nearly the same (both between 26MB/sec and
> 28MB/sec). Maybe I should have done it in single user mode.
> 
> My other hard disc ad1 (it is newer and bigger and faster and more
> furious) behaves better (but I can just try writing via the file
> system (ufs+s) and a final sync): The sync just needs .24sec of
> 18.28 seconds; the read and write speed is nearly the same (about
> 37MB/sec+/-1MB/sec).
> 
> Is there anything else, that could help to find the reason for the
> difference between ad0's speed in R4.11 and R5.3?

I'll be honest here, I don't care much if the speed difference between 
4.X and 5.X is measureable, or whatever.  What I find is a little 
telling of an issue somewhere, is that READS are slower than WRITES! 
This is totally bogus to me - dd'ing a file to a filesystem, then 
umounting should take longer than dd'ing from the disk to /dev/null, it 
nearly every config I can dream up.  Maybe it's the speed at which 
/dev/null can gobble bits (seems highly unlikely!), or maybe GEOM is 
busy doing a check or some routine to data being accessed directly from 
the disk device instead of through a filesystem?  I don't know, but it 
is an issue, and I'm sure we'll get nailed up to a fence in some 
benchmark somewhere if we don't fix it..

Eric



-- 
------------------------------------------------------------------------
Eric Anderson        Sr. Systems Administrator        Centaur Technology
A lost ounce of gold may be found, a lost moment of time never.
------------------------------------------------------------------------