FreeBSD Compiler Benchmark: gcc-base vs. gcc-ports vs. clang
Mehmet Erol Sanliturk
m.e.sanliturk at gmail.com
Sun Mar 13 20:29:59 UTC 2011
On Sun, Mar 13, 2011 at 3:19 PM, Jakub Lach <jakub_lach at mailplus.pl> wrote:
>
>
> Vinícius Zavam wrote:
> >
> >
> > i'm still curious about things like CPUTYPE= and -march= configured as
> > native, gentlemen.
> > is it the "golden egg" to use with our system or not? why "natives"
> > aren't in the benchs?
> >
> > /me feels confused.
> >
> >
> > --
> > Vinícius Zavam
> > profiles.google.com/egypcio
> >
>
> Apparently -march=native would equal -march=core2
> with 65nm generation Core2s, this is not the case with
> Penryns.. But there are none in the test?
>
> However, I agree that testing with -march=native
> would be simpler and more straightforward.
>
> regards,
> - Jakub Lach
> --
>
The compilers Clang and GCC may also be compared with the following design
because ( on the same computer , multiple parameters are measured ) :
Clang Version x
------------------------------------
Repeated Measures --->
p(1) p(2) p(3) p(4) and other parameters
up to p(m)
----- ----- ----- -----
Computer 1 value value value value .....
Computer 2 value value value value .....
.
.
.
Computer n value value value value .....
GCC Version x
------------------------------------
Repeated Measures --->
p(1) p(2) p(3) p(4) and other
parameters up to p(m)
----- ----- ----- -----
Computer n+1 value value value value .....
Computer n+2 value value value value .....
.
.
.
Computer n+n value value value value .....
For each compiler the same number of computers are used ( This is called
balanced design ) .
Evaluation of unbalanced designs may not be available in used statistical
packages , and
theoretically , they are NOT very good .
Here factors are :
(1) Compilers ( CLang , GCC )
(2) Measured parameters as Compilation Parameters
such as
( No optimization )
( Optimization Level 1 )
( Optimization Level 2 )
( Processor Kind 1 )
( Processor Kind 2 ) and
( Code Generation Kind 1 )
( Code Generation Kind 2 ) and/or
any number of other parameters
Number of computers as n should be greater than Number of parameters as m .
Subjects are the computers which no one of them is equal to the other .
Measured parameters are also called treatments .
In statistical analysis packages and
books on experimental designs
this design is called
two-factor experiment with repeated measures on one factor .
Also the other names may be used :
Repeated measures design , or
within-subjects factorial design , or
multifactor experiments having repeated measures on the same elements
Inclusion of two GCC versions into the above table as another compiler may
NOT be very well ,
because GCC compilers are likely VERY CORRELATED with each other ( because
they are using
the same code generator perhaps some patches applied to distinguish the
versions ).
To obtain an idea about correlation strength of the GCC compilers ,
CANONICAL correlation analysis may be used when there are multiple
parameters
( do NOT use two-by-two correlation coefficients when there are more than
two parameters ) , or ,
simple correlation when there are only two parameters ( one for each
compiler ) .
Design is as follows :
GCC Version x GCC version y
--------------------- ---------------------
p(1) P(2) ... p(k) p(1) (p2) ... p(k)
Computer 1 v v v v v v
computer 2 v v v v v v
.
.
.
Computer n v v v v v v
where
p(1) , p(2) , ... , p(k) are the measured parameters ,
k : Number of parameters for each block individually
( There may be different parameter sets , but
for our purposes equivalent parameter sets are required )
n : Number of observations , where each computer should be different
from
the others .
v : Value measured for a parameter
When there is a significant CANONICAL correlation between two compiler
related
values blocks :
(i) it is NOT possible to include the two compilers in the above
repeated measures design because of high collinearity .
(2) Selection of BEST compiler is possible because two compilers are very
similar
( there are no difference between them other than performance level ) .
When the CANONICAL correlation is NOT significant ,
the other GCC compiler may be included
as a third compiler into the repeated measures design .
http://en.wikipedia.org/wiki/Repeated_measures_design
http://en.wikipedia.org/wiki/Category:Experimental_design
http://en.wikipedia.org/wiki/Canonical_correlation
http://en.wikipedia.org/wiki/Category:Multivariate_statistics
Thank you very much .
Mehmet Erol Sanliturk
More information about the freebsd-current
mailing list