From nobody Sat Dec 11 16:53:25 2021 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 3B2C218D953E for ; Sat, 11 Dec 2021 16:53:34 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: from mail-oi1-x236.google.com (mail-oi1-x236.google.com [IPv6:2607:f8b0:4864:20::236]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4JBDPB10jQz4W67 for ; Sat, 11 Dec 2021 16:53:34 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: by mail-oi1-x236.google.com with SMTP id bf8so17455582oib.6 for ; Sat, 11 Dec 2021 08:53:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=fklZhNnLOBordd+EvvBbmSo2ti8QWXGoghEaaSzXWhs=; b=ggKzVgNWAtczt2kAAZu1NOb6nN5owmJiTN9Ud8WzqnJFotexiwWypulsm/gdrPOIY4 VpKVomr7CMk5m+u13XPlWRzxLxOXy8vbv2rPBnKWuohYqupKAAZr+QNpe8203SNIKZ9H h+uLqEqb8oyTvckb8QSj5kt8X4fOdlMu9FxSl0IoYWJ7H+RYt/iVJ612yL6LLVG4J+hQ MoA8Xe97t55ZhkvJEgtRXj1LC7WmMHh14lSThiBKoitp5GB5fhIbjTGKn0Mc4R0HcUBI MOFJG5Zm0VM3XsBXco6X32f8jmsQkSAc0zt1YCt4FeCtFs2csoPZqr+79HL0yZd1elEC hoQg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=fklZhNnLOBordd+EvvBbmSo2ti8QWXGoghEaaSzXWhs=; b=Fv3RcCBzYS1tJhdWKbbrQKU32frz1rr1eqgWsK2B54QaPMv+6WZIRf5opIs9przCMT 30oJw6foYf5dxxykPMVYhY5zwOOxTsbxV9o7lEwsVdeXJmfeZsjLTrEhU0PFtdQu61nq jjuV2WeojlPs88HphdGiK9AtYdxfpdNd+Q6xrH2JlZOqqIz+Knbssbo78gxGh0AI8yjR C18TSuWbYs1ueevx8+ZBi+GQzH/npJm09B3BAC4JjNSL6NlrDWfK32RzxC7p2Y+a5I3c jLzN83H1f8/NjpgGv6lpp33C9neZFlIwSohKfUJYrOkQeE0w6sPdSJfmSZv9jwz4vmqD WLgg== X-Gm-Message-State: AOAM532s6Q7amUK0w95EY+QUrxE/+FRWQp2pqMZP9GFZmaoqvJAvFVhJ 2slCKK+JAUp6MCLOVGD43X3+jrTLQ/g+G1XpIAB3bzmv X-Google-Smtp-Source: ABdhPJya2k16GGinGcRWXDugq0iC4kSxdIcI/jRV5Ua8rKoo6HKpLfaDqASDLn55k6+TteJVyg3vEQO7sZKFkBFuGXk= X-Received: by 2002:aca:3047:: with SMTP id w68mr18414814oiw.75.1639241606931; Sat, 11 Dec 2021 08:53:26 -0800 (PST) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Received: by 2002:ac9:6c8c:0:b0:3e8:92e0:314b with HTTP; Sat, 11 Dec 2021 08:53:25 -0800 (PST) In-Reply-To: References: From: Mateusz Guzik Date: Sat, 11 Dec 2021 17:53:25 +0100 Message-ID: Subject: Re: Benchmarks: FreeBSD 13 vs. NetBSD 9.2 vs. OpenBSD 7 vs. DragonFlyBSD 6 vs. Linux To: Piper H Cc: freebsd-current@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4JBDPB10jQz4W67 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N On 12/11/21, Piper H wrote: > I read this article from Reddit: > https://www.phoronix.com/scan.php?page=article&item=bsd-linux-eo2021&num=1 > > I am surprised to see that the BSD cluster today has much worse performance > than Linux. > What do you think of this? > There is a lot to say here. One has to own up to Linux likely being a little bit (or even more so) faster for some of the legitimate tests. One, there are certain multicore scalability issues compared Linux, which should be pretty mild given the scale (16 cores/32 threads). A more important problem is userspace which fails to take advantage of SIMD instructions for core primitives like memset, memcpy et al. However, if the difference is more than few %, the result is likely bogus. Key thing to do when benchmarking is being able to explain the result, most notably if you run into huge discrepancies. I had a look at the most egregious result -- zstd and spoiler, it is a bug in result reporting in zstd. I got FreeBSD and Linux (Ubuntu Focal) vms running on: Intel(R) Xeon(R) Platinum 8275CL CPU @ 3.00GHz Their zstd test ultimately ends up spawning: zstd -T24 -S -i15 -b19 FreeBSD-12.2-RELEASE-amd64-memstick.img (yes, they compress a ~1GB FreeBSD image). Side note, it does not matter, but I happen to have CURRENT kernel running on the FreeBSD 13 vm right now. [16:37] freebsd13:~ # time zstd -T24 -S -i15 -b19 FreeBSD-12.2-RELEASE-amd64-memstick.img 19#md64-memstick.img :1055957504 -> 692662162 (1.524), 3.97 MB/s ,2156.8 MB/s zstd -T24 -S -i15 -b19 FreeBSD-12.2-RELEASE-amd64-memstick.img 274.10s user 12.90s system 763% cpu 37.602 total In contrast: [16:37] ubuntu:...tem/compress-zstd-1.5.0 (130) # time zstd -T24 -S -i15 -b19 FreeBSD-12.2-RELEASE-amd64-memstick.img 19#md64-memstick.img :1055957504 -> 692662162 (1.524), 60.1 MB/s ,2030.6 MB/s zstd -T24 -S -i15 -b19 FreeBSD-12.2-RELEASE-amd64-memstick.img 328.65s user 3.48s system 850% cpu 39.070 total This is repeatable. If anything, FreeBSD did it *faster*. Yet zstd reports: FreeBSD: 3.97 MB/s ,2156.8 MB/s [total time real time of 37.602 seconds] Linux: 60.1 MB/s ,2030.6 MB/s [total time real time of 39.070 seconds] I don't know what these numbers are supposed to be, but it is pretty clear Phoronix grabs the first one. I'll look into sorting this out some time later. TL;DR don't drink and benchmark -- Mateusz Guzik