Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Stefan Esser]

Am 21.12.2011 06:22, schrieb Ian Smith:
> I find the results on this page very strange, but perhaps indicative:
> 
> http://www.phoronix.com/scan.php?page=article&item=debian_kfreebsd_h210&num=1
> 
> Here we see scant difference in results between Debian running FreeBSD 
> 7.3 or 8.0 or Linux 2.6.32 kernels, yet native FreeBSD 7.3 and 8.0 
> installations apparently run far slower, especially on the gzip test!

You did not expect this, since all user space programs were compiled
from identical sources, as were FreeBSD and kFreeBSD (probably with
minimal deviations in kFreeBSD, which should not affect the results)?

> Does this imply that given the similar kernel speed, Debian GNU userland 
> performs so dramatically better than FreeBSD userland?  Or does it 
> perhaps point to the default tuning of the FreeBSD systems compared to 
> (here) Debian, for these particular tests?  Indeed, `which gzip`?

Well, the answer is quite simple: Just run the Linux binaries on FreeBSD
or kFreeBSD (those compiled for testing Linux performance) and I'm
convinced that you'll find that performance significantly improves.

You did notice, that the 7-zip and gzip binaries were built with
gcc-4.4.4 for Linux and with gcc-4.2.1 for FreeBSD?

And another point: The relative advantage between FreeBSD and Linux is
different on R52 and T61. Might it be the case that gcc-4.4.4 has better
knowledge of the newer CPU in the latter (T61, Core 2 Duo) and optimizes
for it, not for the CPU in the R52 (Pentium-M) anymore?

And apparently 7-zip results are less affected by the compiler version
than the gzip results. This also hints at the compiler as the reason for
the better kFreeBSD and Linux results. (7-zip seems to be less dependent
on the better optimization of the newer gcc, or it does not take as much
advantage from it.)

Funny is the finding, that gzip is measured slower on FreeBSD 7.3 than
8.0 on the Pentium-M, while it is faster on 7.3 on the Core 2 Duo. That
does not match my expectations at all ...

There are no technical reasons, that FreeBSD does not come with a newer
GCC, as probably all in this list know. But OTOH, the newer GCC versions
can easily be installed from a port or package, and thus it would not
have been impossible to compare native binaries compiled with the same
compiler version for all test cases.

> And yes, FreeBSD could sure use some sort of tuning 'profiles' mechanism 
> to be able to preconfigure systems for at least several vastly different 
> types of workload.  Nate Lawson used to talk about this, then in respect 
> to simple 'laptop vs desktop' scenarios, but we've since seen volumes 
> written, mostly in lists but some wikis, parts of the Handbook, guides 
> for performance tuning etc, scarcely accessible to J. Random Installer.  
> A set of tunings for these Phoronix benchmarks might be a good start?

I doubt that tuning is responsible, because kFreeBSD performed better
(with the test programs compiled with gcc-4.4.4). The benchmark measured
just that, the better optimization of the newer gcc version.

Install the port (perhaps an even later gcc version, gcc-4.5 is said to
generate even better code than gcc-4.4) and make it the default compiler
for ports, if you want to take advantage of the more advanced compiler.
The FreeBSD ports system makes that very easy.

BTW: Why don't we build binary packages with a later version of gcc than
what is in the system. This should not cause any GPLv3 violation, and we
could have the userland built with the compiler giving best performance ...

Regards, Stefan