From nobody Sun Dec 19 16:07:34 2021 X-Original-To: bugs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 31B011906178 for ; Sun, 19 Dec 2021 16:07:34 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4JH70P6Vktz3MdY for ; Sun, 19 Dec 2021 16:07:33 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id BEFAB2574D for ; Sun, 19 Dec 2021 16:07:33 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 1BJG7Xp8074406 for ; Sun, 19 Dec 2021 16:07:33 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 1BJG7XnB074405 for bugs@FreeBSD.org; Sun, 19 Dec 2021 16:07:33 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: bugs@FreeBSD.org Subject: [Bug 254040] AMD 5950X hyperthreading strange performance swings Date: Sun, 19 Dec 2021 16:07:34 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 12.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: se@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Bug reports List-Archive: https://lists.freebsd.org/archives/freebsd-bugs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-bugs@freebsd.org MIME-Version: 1.0 ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1639930053; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EW5ifE7aeIpInFSuVxhGdAO9efqehqPh3Dj2Dt0zFjs=; b=p2BlD0aRLLiBZqPGAYGbvddP/kexN5tcTI8NyQ4svpBKJ8CpQvZqkfa+STRlcT+NGuVSB/ Md/JtfXkNipBUkByipta6FNQ/ZjPDU+LPmFS6j4samFyyjffEEVm2ZODTZxLgqa33BZj+R tnpzi9IZwyF1YJO3GjqHXSPTcfbZKGgo1l0lS4h1Ax9FfkhirDKZp06SKBDm8Xu8OzW7ua ZstzROlMTcjL4BBY+fY4g/7MO/0Rzenh0EXT7tIjkzDT4QweHaLYaQyNemhMtMqjVZSHfT jgAGVBXtcH6XrNzMMhkzRBU2oOIz99yuqGGue9DGSSnCVdc/eBaMudjFD0XY5A== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1639930053; a=rsa-sha256; cv=none; b=Lpc8R2SjPSZSUL8znWIA+KtOq5hq3qq+fn1U6eDXNKRSwB6a6ItZ5z8xdTLZiKZPQ+ydEx XIF4ZlU6XxM1+BGD9rEhH5UeMkIR3nwvWL4GxOKOKp74xoNBLBSD9yCS3csjf+LSS8Zl2R BrIrtkgmwwEE7srJb6YaTA457xVe0l0OymImvbmXUfL8oc/7/EZduUBk8q7Lm+6S0Lbdhp 4ULimSjCjGAXJAP4scsV00XOf920Ymi7ds0lL/qs75JXDEBE1QHgvEn6XaR/4veizJH3Pf H9jVVjbLFVoOCJ2ksDxRoZqIwG2dIKruXABQqtuIDKiGER2tocRXIrWafAb++g== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D254040 --- Comment #3 from Stefan E=C3=9Fer --- Update after booting a kernel with SCHED_4BSD: This really appears to be a SCHED_ULE issue, as expected. With SCHED_4BSD t= he results are quite homogeneous for each run: $ t # repeated runs ... 1048576000 bytes transferred in 6.720101 secs (156035746 bytes/sec) s048576000 bytes transferred in 7.037620 secs (148995835 bytes/sec) 1048576000 bytes transferred in 7.031251 secs (149130790 bytes/sec) 1048576000 bytes transferred in 7.032412 secs (149106159 bytes/sec) 1048576000 bytes transferred in 7.202687 secs (145581229 bytes/sec) 1048576000 bytes transferred in 6.918272 secs (151566177 bytes/sec) 1048576000 bytes transferred in 6.391244 secs (164064463 bytes/sec) 1048576000 bytes transferred in 6.686209 secs (156826683 bytes/sec) 1048576000 bytes transferred in 6.668778 secs (157236600 bytes/sec) 1048576000 bytes transferred in 6.914906 secs (151639959 bytes/sec) 1048576000 bytes transferred in 6.835760 secs (153395678 bytes/sec) 1048576000 bytes transferred in 6.755348 secs (155221619 bytes/sec) $ t 4 1048576000 bytes transferred in 7.574533 secs (138434415 bytes/sec) 1048576000 bytes transferred in 7.776473 secs (134839540 bytes/sec) 1048576000 bytes transferred in 7.839487 secs (133755689 bytes/sec) 1048576000 bytes transferred in 7.856730 secs (133462132 bytes/sec) $ t 4 1048576000 bytes transferred in 7.481412 secs (140157499 bytes/sec) 1048576000 bytes transferred in 7.557077 secs (138754182 bytes/sec) 1048576000 bytes transferred in 7.676920 secs (136588110 bytes/sec) 1048576000 bytes transferred in 8.040430 secs (130412922 bytes/sec) $ t 4 1048576000 bytes transferred in 7.484386 secs (140101810 bytes/sec) 1048576000 bytes transferred in 7.581198 secs (138312698 bytes/sec) 1048576000 bytes transferred in 7.710614 secs (135991248 bytes/sec) 1048576000 bytes transferred in 7.736921 secs (135528856 bytes/sec) $ t 16 1048576000 bytes transferred in 9.879846 secs (106132833 bytes/sec) 1048576000 bytes transferred in 10.087562 secs (103947418 bytes/sec) 1048576000 bytes transferred in 10.232516 secs (102474894 bytes/sec) 1048576000 bytes transferred in 10.237664 secs (102423370 bytes/sec) 1048576000 bytes transferred in 10.320563 secs (101600658 bytes/sec) 1048576000 bytes transferred in 10.375758 secs (101060179 bytes/sec) 1048576000 bytes transferred in 10.443859 secs (100401199 bytes/sec) 1048576000 bytes transferred in 10.481844 secs (100037355 bytes/sec) 1048576000 bytes transferred in 10.494019 secs (99921294 bytes/sec) 1048576000 bytes transferred in 10.510178 secs (99767674 bytes/sec) 1048576000 bytes transferred in 10.559435 secs (99302286 bytes/sec) 1048576000 bytes transferred in 10.638978 secs (98559840 bytes/sec) 1048576000 bytes transferred in 10.678294 secs (98196963 bytes/sec) 1048576000 bytes transferred in 10.808183 secs (97016860 bytes/sec) 1048576000 bytes transferred in 11.084706 secs (94596647 bytes/sec) 1048576000 bytes transferred in 11.231343 secs (93361587 bytes/sec) Seems that my BIOS settings were not optimal during the tests documented in Comment 2. After loading "Optimal Default" settings and re-activation of the energy efficiency options I do get the following sysctl output now: $ sysctl dev.cpu.0 dev.cpu.0.temperature: 29,6C dev.cpu.0.cx_method: C1/hlt C2/io dev.cpu.0.cx_usage_counters: 180 70723 dev.cpu.0.cx_usage: 0.25% 99.74% last 9261us dev.cpu.0.cx_lowest: C8 dev.cpu.0.cx_supported: C1/1/1 C2/2/18 dev.cpu.0.freq_levels: 3400/3740 2800/2800 2200/1980 dev.cpu.0.freq: 3400 dev.cpu.0.%parent: acpi0 dev.cpu.0.%pnpinfo: _HID=3DACPI0007 _UID=3D0 _CID=3Dnone dev.cpu.0.%location: handle=3D\_SB_.PLTF.C000 dev.cpu.0.%driver: cpu dev.cpu.0.%desc: ACPI CPU Anyway, after another reboot back to SCHED_ULE and repeating the tests with= new BIOS settings I see: $ t 1 # multiple runs again ... 1048576000 bytes transferred in 7.640816 secs (137233502 bytes/sec) 1048576000 bytes transferred in 6.225996 secs (168418995 bytes/sec) 1048576000 bytes transferred in 4.852763 secs (216078118 bytes/sec) 1048576000 bytes transferred in 4.832574 secs (216980866 bytes/sec) 1048576000 bytes transferred in 4.819031 secs (217590617 bytes/sec) $ t 4 1048576000 bytes transferred in 4.956440 secs (211558309 bytes/sec) 1048576000 bytes transferred in 7.634614 secs (137344998 bytes/sec) 1048576000 bytes transferred in 7.788965 secs (134623280 bytes/sec) 1048576000 bytes transferred in 7.819442 secs (134098574 bytes/sec) $ t 4 1048576000 bytes transferred in 4.853663 secs (216038083 bytes/sec) 1048576000 bytes transferred in 4.857143 secs (215883287 bytes/sec) 1048576000 bytes transferred in 7.721846 secs (135793430 bytes/sec) 1048576000 bytes transferred in 7.792580 secs (134560821 bytes/sec) $ t 4 1048576000 bytes transferred in 4.908570 secs (213621479 bytes/sec) 1048576000 bytes transferred in 7.742029 secs (135439433 bytes/sec) 1048576000 bytes transferred in 7.784341 secs (134703245 bytes/sec) 1048576000 bytes transferred in 7.794096 secs (134534649 bytes/sec) Very similar to the results in the initial bug report ... And it really shows that SCHED_ULE causes the incoherent performance.=20 But the throughput with SCHED_ULE is a lot higher than with SCHED_4BSD, probably due to the immensely higher system overhead of the latter with a l= arge number of cores. While the CPU% with SCHED_ULE is in the order of 2%, with SCHED_4BSD I have seen values up to 60% with 32 parallel tasks. Another observation: WCPU of the bzip2 processes on the kernel with SCHED_4= BSD was displayed as way beyond 100% (in the order of 300% in one run I remembe= r). Since bzip2 is not multi-threaded (AFAIK) this seems to be a wrong measurem= ent. The CPU% for the bzip2 processes is always near 100% with SCHED_ULE. --=20 You are receiving this mail because: You are the assignee for the bug.=