From nobody Wed Feb 02 10:39:36 2022 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id AC7DB17DA50E; Wed, 2 Feb 2022 10:39:37 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4JpdbF1Qyvz4f5g; Wed, 2 Feb 2022 10:39:37 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1643798377; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=VIfUmed5Lw4ah5275uMl8kD+1pcGNoGP0D06Dk0SZ/Y=; b=PdL9vcMkj0V9Z/ZKKs32rdvJdAlO6kV6s+wdvxliI7HcGv+gaKnrnLjKvuDyGEXtvl29vR ZuaY5y9PFHiZDyzFSJq9Zo/e+9ddEydhWqqdi9E4xBqPmGvVPki6ibkss7x3+NBC1eKaLP 1LmyTKP1AKPf1zSpykEwXfNcCJ5Yp+FHoCRZIT1vYMLFBHiEJG0ptLVhPSbuvf++UNEwWV bStdlXmvb6sGTHZKjQCXGkaiTIK1Hk0grB/DAvWxXJfCRDwoclWtLGbjS2x6uZN/bA6Ggt vuKpeTpuZF6cQ8gFwUe5plEYiMJvRc4gFtfO+J1GGEUu0A+74JWBassFl/Q7MA== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 05CEB22775; Wed, 2 Feb 2022 10:39:37 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 212AdaAV090879; Wed, 2 Feb 2022 10:39:36 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 212AdaJO090878; Wed, 2 Feb 2022 10:39:36 GMT (envelope-from git) Date: Wed, 2 Feb 2022 10:39:36 GMT Message-Id: <202202021039.212AdaJO090878@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Konstantin Belousov Subject: git: e7c544216282 - main - amd64: micro-optimize vptopte()/vtopde() further List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-main@freebsd.org X-BeenThere: dev-commits-src-main@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kib X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: e7c54421628231650d61edd90b271e6b5e322e36 Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1643798377; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=VIfUmed5Lw4ah5275uMl8kD+1pcGNoGP0D06Dk0SZ/Y=; b=KNmm/nmkIciXzgduZWNt3h9Ue3bh7Nv6EopRPQiENAMwyp7P1ENInTLcB8hX8nAaeuZ2fq ojGmm6bN0/j4THJI6wuwwx0Me7cUc8WWCvElwHI+XVUFlqINui4pZ8id8PAJWIqyiqAAfa JeP2106uhYTmUqlRE3KCD5f2HVqxaqjVX9sggpbTxapycwq0Y7f8xKj+AgTtvSeNmOhuIl tBaI6VlSJN1cA3BiBdtaFapgo3Kv0IFg1Sfp+Am3cWfLDcaGQSKySlTC+3gsSEUWviUSYW VhSbJkaFUyAUjrHqZOHbeDVTK+z6svz0GSyDgjd2VmEegTjtgl24TOQukFl2rg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1643798377; a=rsa-sha256; cv=none; b=B7g9LVCf51CirfehGH5qhGBwl+EtULsdHlfTPl3aAWs4pvdVxxbXVfONK+neG+Z14CRqxQ oh5n+46a29fQd4htzTy32osPWXCrqVrzKD9cVsdx+tdgof/0GDoPf0kZvmT98VAUZSgX38 oKHWbOQ6hcO6rM/zDBj//TwMHVWqIX4uheqMnD32V/rJ9iZhoACxbtMp/lfmQ8dx4Bx0K2 QqyemHF9Leh6SlbUpCe0Lg0yDnjzozTdsoLV86tJCxe/ysPY+A+t/QlFU9GjcYd2WB2+d/ 0bdT3w+VWgzM2gg8xn7dRNeYA5Li5PmuQSJnSpZMn+HWuJgYHfB0/aTtLW+evg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by kib: URL: https://cgit.FreeBSD.org/src/commit/?id=e7c54421628231650d61edd90b271e6b5e322e36 commit e7c54421628231650d61edd90b271e6b5e322e36 Author: Konstantin Belousov AuthorDate: 2022-01-08 05:09:09 +0000 Commit: Konstantin Belousov CommitDate: 2022-02-02 09:40:04 +0000 amd64: micro-optimize vptopte()/vtopde() further Eliminate shlq $3,address shift after masking of the va is done, which is needed to convert pt_entry_t[] array index into byte offset. Do it by preshifting the mask, and compensating the right shift of va. Suggested by: alc Reviewed by: markj Tested by: pho Sponsored by: The FreeBSD Foundation MFC after: 1 week Differential revision: https://reviews.freebsd.org/D33786 --- sys/amd64/amd64/pmap.c | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/sys/amd64/amd64/pmap.c b/sys/amd64/amd64/pmap.c index 5f72ff3acf08..dafb6acb5c60 100644 --- a/sys/amd64/amd64/pmap.c +++ b/sys/amd64/amd64/pmap.c @@ -1516,28 +1516,28 @@ pmap_pt_page_count_adj(pmap_t pmap, int count) } } -pt_entry_t vtoptem __read_mostly = (1ul << (NPTEPGSHIFT + NPDEPGSHIFT + - NPDPEPGSHIFT + NPML4EPGSHIFT)) - 1; -pt_entry_t *PTmap __read_mostly = P4Tmap; +pt_entry_t vtoptem __read_mostly = ((1ul << (NPTEPGSHIFT + NPDEPGSHIFT + + NPDPEPGSHIFT + NPML4EPGSHIFT)) - 1) << 3; +vm_offset_t PTmap __read_mostly = (vm_offset_t)P4Tmap; PMAP_INLINE pt_entry_t * vtopte(vm_offset_t va) { KASSERT(va >= VM_MAXUSER_ADDRESS, ("vtopte on a uva/gpa 0x%0lx", va)); - return (PTmap + ((va >> PAGE_SHIFT) & vtoptem)); + return ((pt_entry_t *)(PTmap + ((va >> (PAGE_SHIFT - 3)) & vtoptem))); } -pd_entry_t vtopdem __read_mostly = (1ul << (NPDEPGSHIFT + NPDPEPGSHIFT + - NPML4EPGSHIFT)) - 1; -pd_entry_t *PDmap __read_mostly = P4Dmap; +pd_entry_t vtopdem __read_mostly = ((1ul << (NPDEPGSHIFT + NPDPEPGSHIFT + + NPML4EPGSHIFT)) - 1) << 3; +vm_offset_t PDmap __read_mostly = (vm_offset_t)P4Dmap; static __inline pd_entry_t * vtopde(vm_offset_t va) { KASSERT(va >= VM_MAXUSER_ADDRESS, ("vtopde on a uva/gpa 0x%0lx", va)); - return (PDmap + ((va >> PDRSHIFT) & vtopdem)); + return ((pt_entry_t *)(PDmap + ((va >> (PDRSHIFT - 3)) & vtopdem))); } static u_int64_t @@ -2229,12 +2229,12 @@ pmap_bootstrap_la57(void *arg __unused) */ v_pml5[PML5PML5I] = KPML5phys | X86_PG_RW | X86_PG_V | pg_nx; - vtoptem = (1ul << (NPTEPGSHIFT + NPDEPGSHIFT + NPDPEPGSHIFT + - NPML4EPGSHIFT + NPML5EPGSHIFT)) - 1; - PTmap = P5Tmap; - vtopdem = (1ul << (NPDEPGSHIFT + NPDPEPGSHIFT + - NPML4EPGSHIFT + NPML5EPGSHIFT)) - 1; - PDmap = P5Dmap; + vtoptem = ((1ul << (NPTEPGSHIFT + NPDEPGSHIFT + NPDPEPGSHIFT + + NPML4EPGSHIFT + NPML5EPGSHIFT)) - 1) << 3; + PTmap = (vm_offset_t)P5Tmap; + vtopdem = ((1ul << (NPDEPGSHIFT + NPDPEPGSHIFT + + NPML4EPGSHIFT + NPML5EPGSHIFT)) - 1) << 3; + PDmap = (vm_offset_t)P5Dmap; kernel_pmap->pm_cr3 = KPML5phys; kernel_pmap->pm_pmltop = v_pml5;