Re: Benchmarks: FreeBSD 13 vs. NetBSD 9.2 vs. OpenBSD 7 vs. DragonFlyBSD 6 vs. Linux

On 12/11/2021 5:38 AM, David Chisnall wrote:
> While I agree on most of your points, the value of Phoronix is that it tests the default install.
> 
> As an end user, I don’t care that a particular program is twice as fast on a particular Linux distro as it is on FreeBSD because of kernel features, compiler options, or dependency choices.
> 
> I would love to see the base system include the ThinLTO (LLVM IR) .a files so that I can do inlining from libc into my program.  I would love for ports to default to ThinLTO unless they break with it.  Apple flipped that switch a few years ago, so a lot of things that broke with ThinLTO are now fixed.
> 
> The FreeBSD memcpy / memset implementations look like they’re slower than the latest ones, which can give a 5-10% perf boost on some workloads.  LLVM just landed the automemcpy framework, which is designed by some Google folks to synthesises efficient memcpy implementations tailored to different workloads.
> 
> FreeBSD often wins versus glibc-based distros because jemalloc is faster than dlmalloc (the default malloc implementations in FreeBSD libc and glibc, respectively).  I’ve been using snmalloc in my libc for a while and it generally gives me a few percent more perf.  Unfortunately, FreeBSD decided to expose all of the jemalloc non-standard functions from libc, which means I can’t contribute it to upstream without implementing all of those on top of snmalloc or it would be an ABI break.
> 
> It would be great if someone could pick up the Phronix benchmark suite and do some profiling: where is FreeBSD spending more time than Linux?  Are there Linux-specific code paths that hit slow paths on FreeBSD and fast paths on Linux that could have FreeBSD-specific fast paths added (e.g. futex vs _umtx_op)?
> 
> David
> 
> 
>> On 11 Dec 2021, at 10:17, dmilith . <dmilith@gmail.com> wrote:
>>
>> 1. Where are compiler options for BSDs?
>> 2. Why they compare -O2 to -O3 code in some benchmarks? Why they enable
>> fast math in some, and disable it for others?
>> 3. Why they don't mention powerd setup for FreeBSD? By default it may use
>> slowest CPU mode. Did they even load cpufreq kernel module?
>> 4. Did they even care about default FreeBSD mitigations (via sysctl)
>> enabled, or it's only valid for Linuxes? ;)
>> 5. What happened to security and environment details of BSDs?
>>
>> It's kinda known that guys from Phroenix lack basic knowledge of how to do
>> proper performance testing and lack basic knowledge about BSD systems.
>> Nothing new. Would take these results with a grain of salt.
>>
>> On Sat, 11 Dec 2021 at 10:53, beepc.ch <xpetrl@beepc.ch> wrote:
>>
>>>> I am surprised to see that the BSD cluster today has much worse
>>> performance
>>>> than Linux.
>>>> What do you think of this?
>>>
>>> "Default" FreeBSD install setting are quite conservative.
>>> The Linux common distros are high tuned, those benchmark is in my
>>> opinion comparison of apples and oranges.
>>>
>>> Comparing "default" FreeBSD install with "default" Slackware install
>>> would be more interesting, because Slackware builds are at most vanilla.
>>>
>>>
>>
>> -- 
>> Daniel Dettlaff
>> Versatile Knowledge Systems
>> verknowsys.com
> 
> 

I tried to investigate this a bit yesterday.
Specifically, seeing 'zstd' and 'openssl 3.0 sha256' being significantly 
worse on FreeBSD, when especially the latter, should basically have 0 
difference between OSes.

The Phoronix test suite is available as a package for freebsd, but I 
don't know what kind of magic is required to make it actually work.
The compress-zstd-1.5.0 benchmark's install.sh tries to compile zstd 
with BSD make, when the makefile only works for GNU make. A little 
hacking around that, and I managed to run the test, but the results I 
got were in line with Linux, over 2 GB/sec, not the 300 MB/sec their 
result reported.

I've not managed to figure out what they could have done wrong. It 
really does look like for zstd, it was someone only using 1 core on 
FreeBSD, and all cores on every other OS.

-- 
Allan Jude