FreeBSD Compiler Benchmark: gcc-base vs. gcc-ports vs. clang

[Poul-Henning Kamp]

In message <4D7A42CC.8020807 at FreeBSD.org>, Martin Matuska writes:

>But what I can say, e.g. for the Intel Atom processor, if there are
>performance gains in all but one test (that falls 2% behind), generic
>perl code (the routines benchmarked) on this processor is very likely to
>run faster with that setup.

No, actually you cannot say that, unless you run all the tests at
least three times for each compiler(+flag), calculate the average
and standard deviation of all the tests, and see which, if any of
the results are statistically significant.

Until you do that, you numbers are meaningless, because we have no
idea what the signal/noise ratio is.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.