RELENG_4 -> 5 -> 6: significant performance regression

[Dmitry Pryanishnikov]

Hello!

On Fri, 12 May 2006, Kris Kennaway wrote:
>>>>>>                %Sys   %Intr   %Idl
>>>>>> RELENG_6 + rl0      45      40     15
>>>>>> RELENG_6 + fxp0     45      35     20
>>>
>>>>                  %Sys   %Intr   %Idl  "time md5 -t" wall clock time
>>>> RELENG_6 + rl0      34      24     42   1:43
>>>> RELENG_6 + fxp0     30      20     50   1:40
>>
>> is caused by just these:
>>
>> options 	INVARIANTS
>> options 	INVARIANT_SUPPORT
>
> So what is the overall status?  I am not clear what your results are.

  Results for RELENG_6+rl0 are

%Sys   %Intr   %Idl
34      24     42

without INVARIANTS, and

%Sys   %Intr   %Idl
45      40     15

with them. Other options like QUOTA and "makeoptions
CONF_CFLAGS=-fno-builtin" make almost no difference. So, under my test
conditions, the best % of idle CPU time under RELENG_6 is 42%, while under 
RELENG_4 we had

%Sys   %Intr   %Idl
14      14     72

under the same conditions (and with INVARIANTS!) ;(

>> available for application under RELENG_5/6 than under RELENG_4 (under
>> identical load pattern). I ran "time md5 -t" several (3-5 times) just to
>> confirm my assumptions, and results didn't vary more than 3%. So I suppose
>> that ministat isn't necessary in my tests.
>
> Perhaps not when the difference is large, but you need to be very
> careful when differences are below ~10%, because it's easy to make
> incorrect conclusions.

  I agree with you. I would make more measurements if my aim was to determine
which branch between RELENG_5 and _6 to use. But as these results are close
enough, and RELENG_6 is superiour regarding new features (and often 
stability), IMHO there's no point in using RELENG_5 at all. I'm just trying
to understand why performance of RELENG_6 is worse than in RELENG_4 _that 
much_, and whether this sad situation can be improved somehow.

> Kris

Sincerely, Dmitry
-- 
Atlantis ISP, System Administrator
e-mail:  dmitry at atlantis.dp.ua
nic-hdl: LYNX-RIPE