Re: Benchmarks: FreeBSD 13 vs. NetBSD 9.2 vs. OpenBSD 7 vs. DragonFlyBSD 6 vs. Linux

On 2021-12-11 02:38, David Chisnall wrote:
> While I agree on most of your points, the value of Phoronix is that it tests 
> the
> default install.
> 
> As an end user, I don’t care that a particular program is twice as fast on a
> particular Linux distro as it is on FreeBSD because of kernel features, 
> compiler
> options, or dependency choices.
> 
> I would love to see the base system include the ThinLTO (LLVM IR) .a files 
> so that
> I can do inlining from libc into my program.  I would love for ports to 
> default to
> ThinLTO unless they break with it.  Apple flipped that switch a few years 
> ago, so
> a lot of things that broke with ThinLTO are now fixed.
> 
> The FreeBSD memcpy / memset implementations look like they’re slower than 
> the
> latest ones, which can give a 5-10% perf boost on some workloads.  LLVM just
> landed the automemcpy framework, which is designed by some Google folks to
> synthesises efficient memcpy implementations tailored to different 
> workloads.
> 
> FreeBSD often wins versus glibc-based distros because jemalloc is faster 
> than
> dlmalloc (the default malloc implementations in FreeBSD libc and glibc,
> respectively).  I’ve been using snmalloc in my libc for a while and it 
> generally
> gives me a few percent more perf.  Unfortunately, FreeBSD decided to expose 
> all of
> the jemalloc non-standard functions from libc, which means I can’t 
> contribute it
> to upstream without implementing all of those on top of snmalloc or it would 
> be an
> ABI break.
> 
> It would be great if someone could pick up the Phronix benchmark suite and 
> do some
> profiling: where is FreeBSD spending more time than Linux?  Are there
> Linux-specific code paths that hit slow paths on FreeBSD and fast paths on 
> Linux
> that could have FreeBSD-specific fast paths added (e.g. futex vs _umtx_op)?
I think everyones (here) making good points on the comparisons made on/at 
Phronix.
But given that the FreeBSD "default" install adds a fair amount of overhead 
to
elicit good feedback for bugs/failures. It makes it a poor candidate (kernel) 
for
comparing performance. Hell, dmesg(8) even throws a warning saying that 
performance
will be encumbered.
If they knew the BSD basics. They might be able to provide a more meaningful 
comparison.

I'm going to add Creating a BSD vs Linux comparison to the Foundation 
Sponsored Request/poll
recently posted on some of the mailing lists. It'd be great for 
promotion/advertising/evangelism. :-)

-- Chris
> 
> David
> 
> 
>> On 11 Dec 2021, at 10:17, dmilith . <dmilith@gmail.com> wrote:
>> 
>> 1. Where are compiler options for BSDs?
>> 2. Why they compare -O2 to -O3 code in some benchmarks? Why they enable
>> fast math in some, and disable it for others?
>> 3. Why they don't mention powerd setup for FreeBSD? By default it may use
>> slowest CPU mode. Did they even load cpufreq kernel module?
>> 4. Did they even care about default FreeBSD mitigations (via sysctl)
>> enabled, or it's only valid for Linuxes? ;)
>> 5. What happened to security and environment details of BSDs?
>> 
>> It's kinda known that guys from Phroenix lack basic knowledge of how to do
>> proper performance testing and lack basic knowledge about BSD systems.
>> Nothing new. Would take these results with a grain of salt.
>> 
>> On Sat, 11 Dec 2021 at 10:53, beepc.ch <xpetrl@beepc.ch> wrote:
>> 
>>>> I am surprised to see that the BSD cluster today has much worse
>>> performance
>>>> than Linux.
>>>> What do you think of this?
>>> 
>>> "Default" FreeBSD install setting are quite conservative.
>>> The Linux common distros are high tuned, those benchmark is in my
>>> opinion comparison of apples and oranges.
>>> 
>>> Comparing "default" FreeBSD install with "default" Slackware install
>>> would be more interesting, because Slackware builds are at most vanilla.
>>> 
>>> 
>> 
>> --
>> Daniel Dettlaff
>> Versatile Knowledge Systems
>> verknowsys.com