From nobody Sat Dec 31 22:10:40 2022 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4NkxCN5PpMz2nXDF; Sat, 31 Dec 2022 22:10:40 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4NkxCN4WgBz41mF; Sat, 31 Dec 2022 22:10:40 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1672524640; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=8D4NdlBAzfS1Efj6BcO4NkNyxzcwn7Z3hVLRNg0LfsA=; b=bm7MT9D2f4qILrWQ9pOBT1WZY91NoxTUkYjLjZeoOBJS2nCCagByUOMfqbeU+JFihdtWvl GIMNummOyMRV3v917MvCeO89E/AqOv8clwMx5gpAz03qqLmCgA5myq9Od7t8WtjAc2zhNS svwjIIrSR12hIYUnHVhAg29oUt8j49D/A8SsZ3UUOKTtj47bi67hLFOv1fn4yx+/yr5GUi Vtjm9xJH0/7ZMgE8ZuMzv1hWSige+Q8yVjv6Z+cXLTPLOtk15kL7jpOgwbIzrPG4WS9RYT 8o4/+6cCkFnIHMrWiV8R9vg+tnd7BlZ8ViSxt93I7vrEWkk49CW9CT6J1JwnyQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1672524640; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=8D4NdlBAzfS1Efj6BcO4NkNyxzcwn7Z3hVLRNg0LfsA=; b=hVyIS9/Ls5YQsVPK/t5OvYvnCVQ3hmROzWgHW1kf7gumvGJ4zjBlqnX31BRuZqquVsdzWE wBUQ3P+IOH3OKAmWvHAZYF+xokAJ+ka1uS9uqf/ovtOIUJSwa9VXCKY3lUfxBCAbtFg6FQ PBCGoo1eoGUGw/Oa5SMoYsm5lNXKRWTQY7jNog1B3FGdP+tBwA9zQMsDDXoSXxF7RpOAgW Ee5LIhkM7ZWIz4E7+M6kvq6ZiyythjGvX/ZHR4BMvwFlWNBX4HFiXW77bCgUzc1XENfaqN 4nv+8To1MRB7TLWHKbgG8f8VuZKz2G5VZ7QfPoMm749KC9Mmnd7xW+JR9Ptg1A== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1672524640; a=rsa-sha256; cv=none; b=L7z8fk6wSkzMMbFt/gztndXCuLOVX8YvBP2PRVHl82m82B17YFTY9iz4NrWzyZ+OqibQWs BLEczz5gkbgbvNCKFEBcqiY4I5gtqNSW595bVdh5M+OaYIuXIX8dMFAHoCIwsubjL1PrCI LfLk+rWm7wSiIL4sHJktpyACokglqbyCT9VqjNt2S0COt0Vnt5aTXExGgEjnBpC4hSpLyv 5RG17HlT9eJoWZx2mrX85PKpHrgPrxHZJBqBi1Y2gxcfoZKTJdJu8KJUuNxkt5WWW2/cGw 15GvM2lU8O0KGMfS7Ew4a4AdGFgGD6ApTsq6YWYdKrQYmPXbFAHdLVxkcEuhYA== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4NkxCN3chNz14Cs; Sat, 31 Dec 2022 22:10:40 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 2BVMAeUq010669; Sat, 31 Dec 2022 22:10:40 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 2BVMAeZ5010668; Sat, 31 Dec 2022 22:10:40 GMT (envelope-from git) Date: Sat, 31 Dec 2022 22:10:40 GMT Message-Id: <202212312210.2BVMAeZ5010668@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Konstantin Belousov Subject: git: 231d75568f16 - main - Move INVLPG to pmap_quick_enter_page() from pmap_quick_remove_page(). List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-main@freebsd.org X-BeenThere: dev-commits-src-main@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kib X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 231d75568f169fe6adc30373c12512b992c5c672 Auto-Submitted: auto-generated X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by kib: URL: https://cgit.FreeBSD.org/src/commit/?id=231d75568f169fe6adc30373c12512b992c5c672 commit 231d75568f169fe6adc30373c12512b992c5c672 Author: Konstantin Belousov AuthorDate: 2022-12-29 20:48:51 +0000 Commit: Konstantin Belousov CommitDate: 2022-12-31 22:09:46 +0000 Move INVLPG to pmap_quick_enter_page() from pmap_quick_remove_page(). If processor prefetches neighboring TLB entries to the one being accessed (as some have been reported to do), then the spin lock does not prevent the situation described in the "AMD64 Architecture Programmer's Manual Volume 2: System Programming" rev. 3.23, "7.3.1 Special Coherency Considerations". Reported and reviewed by: alc Sponsored by: The FreeBSD Foundation MFC after: 1 week Differential revision: https://reviews.freebsd.org/D37770 --- sys/amd64/amd64/pmap.c | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-) diff --git a/sys/amd64/amd64/pmap.c b/sys/amd64/amd64/pmap.c index 07a00963004b..2d2e673e7622 100644 --- a/sys/amd64/amd64/pmap.c +++ b/sys/amd64/amd64/pmap.c @@ -10423,6 +10423,13 @@ pmap_quick_enter_page(vm_page_t m) return (PHYS_TO_DMAP(paddr)); mtx_lock_spin(&qframe_mtx); KASSERT(*vtopte(qframe) == 0, ("qframe busy")); + + /* + * Since qframe is exclusively mapped by us, and we do not set + * PG_G, we can use INVLPG here. + */ + invlpg(qframe); + pte_store(vtopte(qframe), paddr | X86_PG_RW | X86_PG_V | X86_PG_A | X86_PG_M | pmap_cache_bits(kernel_pmap, m->md.pat_mode, 0)); return (qframe); @@ -10435,14 +10442,6 @@ pmap_quick_remove_page(vm_offset_t addr) if (addr != qframe) return; pte_store(vtopte(qframe), 0); - - /* - * Since qframe is exclusively mapped by - * pmap_quick_enter_page() and that function doesn't set PG_G, - * we can use INVLPG here. - */ - invlpg(qframe); - mtx_unlock_spin(&qframe_mtx); }