Possible evidence of performance regression for 8.1-S (vs. 7.1)
david at catwhisker.org
Fri Oct 22 23:29:08 UTC 2010
On Wed, Oct 20, 2010 at 04:50:10PM -0700, Julian Elischer wrote:
> try the 7.x machine but running the 8.x kernel.. i.e. change nothing,
> but boot the new kernel.
I just started that test, to run over the weekend.
I did run another test (3 iterations) with the 7.1 OS (kernel+userland)
on the physical machine that normally runs 8.x. Here is a summary
of the earlier results, along with the new ones:
start stop real user sys host os
1287525363 1287546846 21483.13 82628.20 21703.09 ref-7.x 7.1-R+
1287548005 1287569100 21094.63 82853.19 22185.02 ref-7.x 7.1-R+
1287570300 1287591371 21071.33 82756.81 21943.22 ref-7.x 7.1-R+
1287592592 1287614103 21511.23 82637.30 21849.90 ref-7.x 7.1-R+
1287615323 1287636770 21446.42 82715.81 21708.97 ref-7.x 7.1-R+
1287436357 1287461948 25590.99 81502.22 18115.07 ref-8.x 8.1-S at r214029
1287462797 1287488766 25969.26 81452.14 17920.14 ref-8.x 8.1-S at r214029
1287489641 1287515287 25645.84 81548.40 18256.52 ref-8.x 8.1-S at r214029
1287516151 1287541481 25329.64 81546.23 18294.10 ref-8.x 8.1-S at r214029
1287542355 1287568599 26244.59 81431.47 17902.39 ref-8.x 8.1-S at r214029
1287710312 1287732046 21733.20 82688.01 22108.95 ref-8.x 7.1-R+
1287733360 1287754549 21188.88 82869.09 21890.83 ref-8.x 7.1-R+
1287755881 1287777566 21684.09 82772.50 21933.74 ref-8.x 7.1-R+
I only ran 3 tests, as the purpose was merely a reality check --
to verify that we weren't actually seeing differences in hardware
between the two machines.
I believe that the above results provide adequate reason to believe
that hardware is not the underlying issue: the 7.1 results on the
ref-8.x machine are just about identical to the 7.1 results on the
ref-7.x machine, and both are noticably different from the 8.1
> >FWIW, the workload is fairly CPU intensive during most of the run; the
> >I/O done during (most of) the test appears to be very light, and the
> >memory used is fairly modest. In each of the test machines, I have
> >turned off HTT (HyperThreading Technology); hw.ncpu reports 8 for each.
> try with HTT on for modern hardware..
OK, I'll try that experiment at some point when I have the time and
other resources, but what I'm trying to point out is that I believe
I'm seeing something that appears to be a performance regression,
comparing 7.1 (+ a few patches that were later committed to stable/7)
to stable/8 @r214029, on identically configured hardware running
the same workload (that happens to be critical for my employer).
FWIW, I tried comparing the 7.1 (as above) vs. stable/7 a couple
of months ago, and found no statistically significant difference
in performance. Thus, I believe it would be possible for someone
else to reproduce my results, comparing a fasirly recent stable/7
to a fairly recent stable/8. If someone does try that (with a
workload primarily composed of building software), and gets results
that differ significantly from what I'm reporting, I would appreciate
an opportunity to compare notes, as it would then be apparent that
I've managed to damage my stable/8 environment in some way.
On the other hand, if someone else tries that, and gets results
that corroborate what I'm seeing, we may have some work to do to
get stable/8 whipped into shape before 8.2-RELEASE (for which the
code freeze is currently scheduled for 28 Nov).
Granted, performance isn't the only thing that drives releases, but I
know that for the developers I support, having an 18% increase in
elapsed time to perform a critical task, merely because of an "upgrade"
to the OS, would be ... poorly received. And there's no way I could
recommend doing that, unless the alternatives were demonstrably worse.
I'm a bit concerned, here.
David H. Wolfskill david at catwhisker.org
Depriving a girl or boy of an opportunity for education is evil.
See http://www.catwhisker.org/~david/publickey.gpg for my public key.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Size: 196 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-performance/attachments/20101022/65fd4cf6/attachment.pgp
More information about the freebsd-performance