From nobody Thu Jun 30 01:14:01 2022 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id A057187B574; Thu, 30 Jun 2022 01:14:01 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4LYL2K3yLQz4VTP; Thu, 30 Jun 2022 01:14:01 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1656551641; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=T6DXIUVntGC6gHvbYTiEL44J7WWP9RpJ+FtGH+0nQBw=; b=Rd8LWDfS15p/wPXUqb4Jdj9mZiTLpJXoEyqw+MGmR377Pf/CtQgzO9RymJAPaufJAzwED0 TGbCZZArv5Q5rNkViYA8Lg+RRlAZvzodre7BhScmudMSbvTYI6CTnV2gYVi0DkpMJakL0V ehu9RmspOsGO0HYwr1RBa2xBOEI7I9FcgsWruY/pKT4wX7az+u12XMQP3r8pk9L0gZC8l3 4pswBFf9Oe72a9mFmi7LDbx+K15cWAordp8IIONEeahK4TeysGOtnItylRoMFRzrV+UBm6 qeme/irgPg8ierP5OUOboB+wAXtspVbFM4ZnxGPOr+qkG4DDl8WvHw+u+CA2PA== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 678861CBAA; Thu, 30 Jun 2022 01:14:01 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 25U1E1qb097628; Thu, 30 Jun 2022 01:14:01 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 25U1E1bj097627; Thu, 30 Jun 2022 01:14:01 GMT (envelope-from git) Date: Thu, 30 Jun 2022 01:14:01 GMT Message-Id: <202206300114.25U1E1bj097627@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Alexander Motin Subject: git: 558e8b6adb89 - stable/13 - amd64: Stop using REP MOVSB for backward memmove()s. List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mav X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 558e8b6adb89faeb4c31faf14e88859d8c14c61b Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1656551641; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=T6DXIUVntGC6gHvbYTiEL44J7WWP9RpJ+FtGH+0nQBw=; b=CaLUDw13MOlSm3Zm44899oCONpPhYbKN7NQdvPR920jtmnoJcnIeap8f3nqP9OoW31ldGl jgHkNcZS7O1OcwrJBG60F5w2qt9oxt37kSqZQz8HrtMFTTsp/GWXUIegxX4kWz8jkoaEOt qTMxJLanVdcPOFQ+/ByO/+IVuObf35sKUStElHqGsuIBh6m+2nmgLeGv4kgJeZOw47CviM PuzLGyKjEBrq+cAKtUEy3obaywEsMF9d6SOtEAPFt5LCFXRtDaX81p/x5eWeunGsjdPvtk IsCu62m8rGD0ICuiYzJMuGlOyu6Ei8KjNMuQKHTF2euLxpuYS/90vxRpsXfaxQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1656551641; a=rsa-sha256; cv=none; b=VKUMSnFvYWgF9kMvNun7xImZI5W8B88d4egixrZPbk1WlHcLEjohC7LDgicozsv3o4ofw+ UCIyZafHobU+OL9p8iwbSD4SSuTp007+PKt8Ka1CPSOPX9jkotZTOUumxSC41CeU5mMcxS 7+MS0rJEQ0vUN9NnhwnjlfiVAGPjrX2JgT4gqfU1w6vbMoGKLJJY1oSwAb8v4YmZgK+9Zc iSP/Oo4NWeWd4ZO968l22V8KBtp5prVogagFC8pCImw0kO/595zva8ArLnRUO4B/Dd4wPN YB4lF75zJwccI0gjqSfIZ6CCz6rpSconnQCtW8Rfa0JZVsC4jHqcoGKDTDXwnw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch stable/13 has been updated by mav: URL: https://cgit.FreeBSD.org/src/commit/?id=558e8b6adb89faeb4c31faf14e88859d8c14c61b commit 558e8b6adb89faeb4c31faf14e88859d8c14c61b Author: Alexander Motin AuthorDate: 2022-06-16 18:51:50 +0000 Commit: Alexander Motin CommitDate: 2022-06-30 01:13:51 +0000 amd64: Stop using REP MOVSB for backward memmove()s. Enhanced REP MOVSB feature of CPUs starting from Ivy Bridge makes REP MOVSB the fastest way to copy memory in most of cases. However Intel Optimization Reference Manual says: "setting the DF to force REP MOVSB to copy bytes from high towards low addresses will expe- rience significant performance degradation". Measurements on Intel Cascade Lake and Alder Lake, same as on AMD Zen3 show that it can drop throughput to as low as 2.5-3.5GB/s, comparing to ~10-30GB/s of REP MOVSQ or hand-rolled loop, used for non-ERMS CPUs. This patch keeps ERMS use for forward ordered memory copies, but removes it for backward overlapped moves where it does not work. This is just a cosmetic sync with kernel, since libc does not use ERMS at this time. Reviewed by: mjg MFC after: 2 weeks (cherry picked from commit f22068d91bf53696ee13a69685e809d35776ec3f) --- lib/libc/amd64/string/memmove.S | 8 -------- 1 file changed, 8 deletions(-) diff --git a/lib/libc/amd64/string/memmove.S b/lib/libc/amd64/string/memmove.S index 3d75ff45c798..ea92cb18782a 100644 --- a/lib/libc/amd64/string/memmove.S +++ b/lib/libc/amd64/string/memmove.S @@ -274,13 +274,6 @@ __FBSDID("$FreeBSD$"); ALIGN_TEXT 2256: std -.if \erms == 1 - leaq -1(%rdi,%rcx),%rdi - leaq -1(%rsi,%rcx),%rsi - rep - movsb - cld -.else leaq -8(%rdi,%rcx),%rdi leaq -8(%rsi,%rcx),%rsi shrq $3,%rcx @@ -290,7 +283,6 @@ __FBSDID("$FreeBSD$"); movq %rdx,%rcx andb $7,%cl jne 2004b -.endif \end ret .endif