Re: Benchmarks: FreeBSD 13 vs. NetBSD 9.2 vs. OpenBSD 7 vs. DragonFlyBSD 6 vs. Linux
Date: Sat, 11 Dec 2021 16:53:25 UTC
On 12/11/21, Piper H <potthua@gmail.com> wrote: > I read this article from Reddit: > https://www.phoronix.com/scan.php?page=article&item=bsd-linux-eo2021&num=1 > > I am surprised to see that the BSD cluster today has much worse performance > than Linux. > What do you think of this? > There is a lot to say here. One has to own up to Linux likely being a little bit (or even more so) faster for some of the legitimate tests. One, there are certain multicore scalability issues compared Linux, which should be pretty mild given the scale (16 cores/32 threads). A more important problem is userspace which fails to take advantage of SIMD instructions for core primitives like memset, memcpy et al. However, if the difference is more than few %, the result is likely bogus. Key thing to do when benchmarking is being able to explain the result, most notably if you run into huge discrepancies. I had a look at the most egregious result -- zstd and spoiler, it is a bug in result reporting in zstd. I got FreeBSD and Linux (Ubuntu Focal) vms running on: Intel(R) Xeon(R) Platinum 8275CL CPU @ 3.00GHz Their zstd test ultimately ends up spawning: zstd -T24 -S -i15 -b19 FreeBSD-12.2-RELEASE-amd64-memstick.img (yes, they compress a ~1GB FreeBSD image). Side note, it does not matter, but I happen to have CURRENT kernel running on the FreeBSD 13 vm right now. [16:37] freebsd13:~ # time zstd -T24 -S -i15 -b19 FreeBSD-12.2-RELEASE-amd64-memstick.img 19#md64-memstick.img :1055957504 -> 692662162 (1.524), 3.97 MB/s ,2156.8 MB/s zstd -T24 -S -i15 -b19 FreeBSD-12.2-RELEASE-amd64-memstick.img 274.10s user 12.90s system 763% cpu 37.602 total In contrast: [16:37] ubuntu:...tem/compress-zstd-1.5.0 (130) # time zstd -T24 -S -i15 -b19 FreeBSD-12.2-RELEASE-amd64-memstick.img 19#md64-memstick.img :1055957504 -> 692662162 (1.524), 60.1 MB/s ,2030.6 MB/s zstd -T24 -S -i15 -b19 FreeBSD-12.2-RELEASE-amd64-memstick.img 328.65s user 3.48s system 850% cpu 39.070 total This is repeatable. If anything, FreeBSD did it *faster*. Yet zstd reports: FreeBSD: 3.97 MB/s ,2156.8 MB/s [total time real time of 37.602 seconds] Linux: 60.1 MB/s ,2030.6 MB/s [total time real time of 39.070 seconds] I don't know what these numbers are supposed to be, but it is pretty clear Phoronix grabs the first one. I'll look into sorting this out some time later. TL;DR don't drink and benchmark -- Mateusz Guzik <mjguzik gmail.com>