Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Pieter de Goeje]

Op 15-12-2011 8:32, O. Hartmann schreef:
> Just saw this shot benchmark on Phoronix dot com today:
>
> http://www.phoronix.com/scan.php?page=news_item&px=MTAyNzA
>
> It may be worth to discuss the sad performance of FBSD in some parts of
> the benchmark. A difference of a factor 10 or 100 is simply far beyond
> disapointing, it is more than inacceptable and by just reading those
> benchmarks, I'd like to drop thinking of using FreeBSD even as a backend
> server in scientific and business environments. In detail, some of the
> SciMark benches look disappointing. The overall image can't help over
> the fact that in C-Ray FreeBSD is better performing.
>
>  From the compiler, I'd like say there couldn't be a drop of more than 10
> - 15% in performance - but not 10 or 100 times.
>
> I'm just thinking about the discussion of SCHED_ULE and all the saur
> spots we discussed when I stumbled over the test.
>
> Regards,
> Oliver
Detailed results here:
http://openbenchmarking.org/result/1112113-AR-ORACLELIN37

As usual, the phoronix benchmarks are very misleading.
1) The linked benchmarks were not run on the same hardware. Hardware is 
close but not completely equal; for instance different brands of disks 
were used.
2) They didn't use the same compiler. This is really bad and _can_ lead 
to more than a factor 2 performance difference. Especially in 
"scientific" programs where (auto) vectorization is very important. Why 
on earth the benchmarker was too lazy to install a more recent GCC I 
have no idea.

Of all the benchmarks shown only the disk benchmarks are interesting, 
because they actually stress the system. Unfortunately they screwed that 
up too because they were performed on ZFS instead of the default, plain 
UFS which is a lot more like EXT4 in terms of functionality.

The rest are pure CPU bound userspace workloads and I bet that if they 
were performed using the same compiler, similar results would've been 
achieved (barring any major VM differences). In any case we would've 
been able to actually compare FreeBSD vs Oracle Linux instead of GCC 4.5 
vs 4.2. Now they are useless.

I'm sorry if this mail sounds a bit harsh but I'm tired of seeing 
phoronix making the same elementary mistakes again and again even after 
these have been pointed out years ago.

- Pieter