From nobody Mon Nov 29 14:21:05 2021 X-Original-To: dev-commits-src-branches@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 8DE4818B035A; Mon, 29 Nov 2021 14:21:07 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4J2nZp2stjz3h91; Mon, 29 Nov 2021 14:21:06 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 1AEB623B78; Mon, 29 Nov 2021 14:21:05 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 1ATEL5nd056027; Mon, 29 Nov 2021 14:21:05 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 1ATEL5V9056026; Mon, 29 Nov 2021 14:21:05 GMT (envelope-from git) Date: Mon, 29 Nov 2021 14:21:05 GMT Message-Id: <202111291421.1ATEL5V9056026@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mark Johnston Subject: git: e0cee46f5800 - stable/13 - amd64: Initialize kernel_pmap's active CPU set to all_cpus List-Id: Commits to the stable branches of the FreeBSD src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-branches List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-branches@freebsd.org X-BeenThere: dev-commits-src-branches@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: markj X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: e0cee46f5800071c105955c530ae91afa2cca002 Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1638195667; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Ou4ZDkEta9aSOWh8fLDmBMGlB3ahjI41MJBlitTXBgI=; b=vBKcAEMMHqNF0RvsWH79VYyCulBFf0aermX5LWpliR1dqF/ulyjyixMFQevi6XzOlh9avS HQHOugexCOQ5IEGGXaJEBJnIVK+8sEPTyOE3BnI7RyWDqEwvMRBtSWjgcgwASZmh3Dj6Co SFnWMfO756o2sJ/Q6EulcqUgsIDyqibjKS2467WLJOV9TY2OA7/f476TCQ/IAc9ef+US9W FFSqyS8O27+ZuP6T2P66JV236RiR4v9fQMXeHrlHmwgvbe0HiZnkuk2et5iCOzkvvhmSPD z/mAmsQQtmxY62M1v32r/wsxIDpGWi2UZSMrbejY0SWvskeRHuy4ND6JqEU7HA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1638195667; a=rsa-sha256; cv=none; b=INW0vSFosNzBfF4IzL7VJGR1vvWL7S1GOknzERiVz2/+N1xwXU61pij8s5Vdn9Yhke4i/s S4d15N8R6MOJ7JXQ/y5gKc15Gb8igY8LZaZZVIU5VFITuPFAuo8q902Oa6XKevULM78Zl8 VsApYstXQ2IpXbPV2kfvgZRIDkSvfloC/TuLR8AtrQXz0c+mE+nrcZP8UbJYbYbx4Dhk7U Nl476GMhQ9kozYYv1fiEgeNu3yc90xkR1jzoshkQByJQ0HrRYrB7bWd9a2kDDDfu4Z95Yz 9WUHLTT5z1bzaKn3TSt1IIS0a/76DP2U5Vz8jG34uEvdWPsBcMOBXnuxw1rwzw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch stable/13 has been updated by markj: URL: https://cgit.FreeBSD.org/src/commit/?id=e0cee46f5800071c105955c530ae91afa2cca002 commit e0cee46f5800071c105955c530ae91afa2cca002 Author: Mark Johnston AuthorDate: 2021-11-15 17:41:24 +0000 Commit: Mark Johnston CommitDate: 2021-11-29 14:17:08 +0000 amd64: Initialize kernel_pmap's active CPU set to all_cpus This is in preference to simply filling the cpuset, and allows the conditional in pmap_invalidate_cpu_mask() to be elided. Also export pmap_invalidate_cpu_mask() outside of pmap.c for use in a subsequent commit. Suggested by: kib Reviewed by: alc, kib Tested by: pho Sponsored by: The FreeBSD Foundation (cherry picked from commit 71e6e9da225aede95f6813a0fcf886538d0da9fe) --- sys/amd64/amd64/pmap.c | 22 ++++++++++++++-------- sys/amd64/include/pmap.h | 6 ++++++ 2 files changed, 20 insertions(+), 8 deletions(-) diff --git a/sys/amd64/amd64/pmap.c b/sys/amd64/amd64/pmap.c index 524b40ed2fee..986a1b670d60 100644 --- a/sys/amd64/amd64/pmap.c +++ b/sys/amd64/amd64/pmap.c @@ -1924,11 +1924,16 @@ pmap_bootstrap(vm_paddr_t *firstaddr) kernel_pmap->pm_pmltop = kernel_pml4; kernel_pmap->pm_cr3 = KPML4phys; kernel_pmap->pm_ucr3 = PMAP_NO_CR3; - CPU_FILL(&kernel_pmap->pm_active); /* don't allow deactivation */ TAILQ_INIT(&kernel_pmap->pm_pvchunk); kernel_pmap->pm_stats.resident_count = res; kernel_pmap->pm_flags = pmap_flags; + /* + * The kernel pmap is always active on all CPUs. Once CPUs are + * enumerated, the mask will be set equal to all_cpus. + */ + CPU_FILL(&kernel_pmap->pm_active); + /* * Initialize the TLB invalidations generation number lock. */ @@ -2968,12 +2973,6 @@ pmap_invalidate_ept(pmap_t pmap) smr_wait(pmap->pm_eptsmr, goal); } -static cpuset_t -pmap_invalidate_cpu_mask(pmap_t pmap) -{ - return (pmap == kernel_pmap ? all_cpus : pmap->pm_active); -} - static inline void pmap_invalidate_preipi_pcid(pmap_t pmap) { @@ -10808,7 +10807,14 @@ pmap_pti_init(void) pti_finalized = true; VM_OBJECT_WUNLOCK(pti_obj); } -SYSINIT(pmap_pti, SI_SUB_CPU + 1, SI_ORDER_ANY, pmap_pti_init, NULL); + +static void +pmap_cpu_init(void *arg __unused) +{ + CPU_COPY(&all_cpus, &kernel_pmap->pm_active); + pmap_pti_init(); +} +SYSINIT(pmap_cpu, SI_SUB_CPU + 1, SI_ORDER_ANY, pmap_cpu_init, NULL); static pdp_entry_t * pmap_pti_pdpe(vm_offset_t va) diff --git a/sys/amd64/include/pmap.h b/sys/amd64/include/pmap.h index bd6a8c006813..14ff4cf3cde9 100644 --- a/sys/amd64/include/pmap.h +++ b/sys/amd64/include/pmap.h @@ -535,6 +535,12 @@ void pmap_kasan_enter(vm_offset_t); void pmap_kmsan_enter(vm_offset_t); #endif +static __inline cpuset_t +pmap_invalidate_cpu_mask(pmap_t pmap) +{ + return (pmap->pm_active); +} + #endif /* _KERNEL */ /* Return various clipped indexes for a given VA */