From nobody Wed Dec 22 12:49:16 2021 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 7941E1909CDF for ; Wed, 22 Dec 2021 12:49:26 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (turbocat.net [IPv6:2a01:4f8:c17:6c4b::2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4JJtSQ26xXz3tDS; Wed, 22 Dec 2021 12:49:26 +0000 (UTC) (envelope-from hps@selasky.org) Received: from [10.36.2.165] (unknown [178.17.145.105]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 4150D2601E9; Wed, 22 Dec 2021 13:49:24 +0100 (CET) Message-ID: <6dd1ef9f-8f44-a2f1-6377-d2b51fde4d43@selasky.org> Date: Wed, 22 Dec 2021 13:49:16 +0100 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:91.0) Gecko/20100101 Thunderbird/91.4.1 Subject: Re: observations on Ryzen 5xxx (Zen 3) processors Content-Language: en-US To: Andriy Gapon , FreeBSD Current References: From: Hans Petter Selasky In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4JJtSQ26xXz3tDS X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-Spam: Yes X-ThisMailContainsUnwantedMimeParts: N On 12/22/21 13:42, Andriy Gapon wrote: > > There have been some reports on strange / unexpected things with Ryzen > 5xxx processors.  I think I have seen 5950X, 5900X and 5800X mentioned, > not sure about others. > > Since I have 5800X myself I looked into a couple of issues that have > straightforward demonstrators.  I would like to share my findings and > observations on those issues. > > Issue 1.  High wake-up latency for CPU idle states. > > This seems to be related to the so called CC6 idle state. > The official information on it is very sparse. > The state is not explicitly exposed to the OS, at least, though ACPI > interfaces that FreeBSD currently supports. > > In my tests I see that if all logical processors enter an idle state > then an external interrupt can be delayed by 500+ us.  Specifically, I > observed this with an MSI-X interrupt from a discrete network chip. > Interrupts from internal components seem to be affected as well, but to > a lesser degree. > > The deep state in question can be entered regardless of whether C2 (via > I/O) is enabled, C1 (via hlt) is sufficient.  In fact, with > machdep.idle=hlt it works the same. > The state is not entered if at least one logical CPU is not idle. > The state is not entered if machdep.idle=mwait is used.  Apparently, the > processors do not attempt to automatically enter as deep idle modes with > mwait as they do with hlt. > Finally, the state is not entered if zenstates.py utility is used to > disable C6 / CC6 state via an undocumented (publicly) MSR. > > For me personally that state does not cause any annoyances but anyone > who experiences problems related to "stuttering", "jitter", latency > might want to look into this. > > Issue 2.  Uneven performance of CPU intensive tasks, especially with > SCHED_ULE, when SMT is enabled. > > I found out that at least on my hardware all even numbered logical CPUs > can perform much better than odd numbered logical CPUs.  It seems that > hardware threads within a core are not equal.  Maybe this is related to > ability to use boosted frequencies, but maybe something else, I am not > sure. > From a brief look at the ULE code it looks that the selection of a hw > thread within a core is intentionally random when all other things are > equal. > I suspect that the hardware + firmware may actually describe that > performance disparity via ACPI CPPC (_CPC object, etc), but right now we > do not support querying that or making use of it. > > > It would interesting to see if other owners of similar processors can > confirm or provide counter-examples to my observations. > > Simple tests for issue 1: > - ping a host attached to the same switch (so, with very low expected > latency) > - ping 127.0.0.1 > > For issue 2: take some CPU intensive single-threaded task and bind it > (with cpuset -l) to different logical CPUs.  Multiple such tasks can be > run concurrently on different logical CPUs. > > References: > - > https://forums.freebsd.org/threads/variable-ping-latency-on-ryzen-setup.82791/ > > - https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=256594 > - https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254040 > - https://github.com/r4m0n/ZenStates-Linux > - https://github.com/meowthink/ZenStates-FreeBSD --  has a bug >   - https://github.com/avg-I/ZenStates-FreeBSD -- has a fix > - https://www.kernel.org/doc/html/latest/admin-guide/acpi/cppc_sysfs.html > - https://static.linaro.org/connect/lvc21/presentations/lvc21-219.pdf > - > https://uefi.org/specs/ACPI/6.4/14_Platform_Communications_Channel/Platform_Comm_Channel.html > > Hi, I've seen exactly the same thing. See older FreeBSD-current thread: "AMD Ryzen 5 3400G with Radeon Vega Graphics" I just put: machdep.idle=spin In /boot/loader.conf for now. --HPS