Re: Benchmarks: FreeBSD 13 vs. NetBSD 9.2 vs. OpenBSD 7 vs. DragonFlyBSD 6 vs. Linux

On 12/11/21, Mateusz Guzik <mjguzik@gmail.com> wrote:
> On 12/11/21, Piper H <potthua@gmail.com> wrote:
>> I read this article from Reddit:
>> https://www.phoronix.com/scan.php?page=article&item=bsd-linux-eo2021&num=1
>>
>> I am surprised to see that the BSD cluster today has much worse
>> performance
>> than Linux.
>> What do you think of this?
>>
>
> There is a lot to say here.
>
> One has to own up to Linux likely being a little bit (or even more so)
> faster for some of the legitimate tests. One, there are certain
> multicore scalability issues compared Linux, which should be pretty
> mild given the scale (16 cores/32 threads). A more important problem
> is userspace which fails to take advantage of SIMD instructions for
> core primitives like memset, memcpy et al. However, if the difference
> is more than few %, the result is likely bogus. Key thing to do when
> benchmarking is being able to explain the result, most notably if you
> run into huge discrepancies.
>
> I had a look at the most egregious result -- zstd and spoiler, it is a
> bug in result reporting in zstd.
>
> I got FreeBSD and Linux (Ubuntu Focal) vms running on:
> Intel(R) Xeon(R) Platinum 8275CL CPU @ 3.00GHz
>
> Their zstd test ultimately ends up spawning: zstd -T24 -S -i15 -b19
> FreeBSD-12.2-RELEASE-amd64-memstick.img (yes, they compress a ~1GB
> FreeBSD image).
>
> Side note, it does not matter, but I happen to have CURRENT kernel
> running on the FreeBSD 13 vm right now.
>
> [16:37] freebsd13:~ # time zstd -T24 -S -i15 -b19
> FreeBSD-12.2-RELEASE-amd64-memstick.img
> 19#md64-memstick.img :1055957504 -> 692662162 (1.524),  3.97 MB/s ,2156.8
> MB/s
> zstd -T24 -S -i15 -b19 FreeBSD-12.2-RELEASE-amd64-memstick.img
> 274.10s user 12.90s system 763% cpu 37.602 total
>
> In contrast:
>
> [16:37] ubuntu:...tem/compress-zstd-1.5.0 (130) # time zstd -T24 -S
> -i15 -b19 FreeBSD-12.2-RELEASE-amd64-memstick.img
> 19#md64-memstick.img :1055957504 -> 692662162 (1.524),  60.1 MB/s ,2030.6
> MB/s
> zstd -T24 -S -i15 -b19 FreeBSD-12.2-RELEASE-amd64-memstick.img
> 328.65s user 3.48s system 850% cpu 39.070 total
>
> This is repeatable. If anything, FreeBSD did it *faster*. Yet zstd reports:
> FreeBSD: 3.97 MB/s ,2156.8 MB/s [total time real time of  37.602 seconds]
> Linux:  60.1 MB/s ,2030.6 MB/s [total time real time of 39.070 seconds]
>
> I don't know what these numbers are supposed to be, but it is pretty
> clear Phoronix grabs the first one.
>
> I'll look into sorting this out some time later.
>

So I cloned https://github.com/facebook/zstd/ and got the v1.4.8 tag,
as currently imported into FreeBSD. The diff is pretty minimal and
deals with exposing extra symbols.

zstd directly compiled from that source (with mere gmake) correctly
shows 2-digit MB speeds, so it has to be something in the FreeBSD
build which ends up messing with it. I ran out of curiosity at this
point (and more so time) at this point, but I invite someone else to
get to the bottom of this.

Bottom line though: there is no zstd performance problem on FreeBSD.

-- 
Mateusz Guzik <mjguzik gmail.com>