From nobody Thu May 11 13:32:58 2023 X-Original-To: dev-commits-src-branches@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4QHCWZ6BTdz4BFKH; Thu, 11 May 2023 13:32:58 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4QHCWZ55lKz4GMF; Thu, 11 May 2023 13:32:58 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1683811978; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=GTa6a2eauZ6GG/IHWd8snoqDW1CfWJ+6oiXPFTyVHBo=; b=enaQ2EXC7iM6DNyGbB4VSjZiXSMtPcYGQN/6vHYauCkv/BmT+DVmw3vMNUWurUOP9CuLeC QwbA34Tj6Uk5iwsU9kL09O6Oo+RqRNih1xts+lJldMhyTYQGZf7OFwtKSwa4LQh7sUz3rc hJBMJL7eAztQZ8xMKDmFFRt7HxNERvQ4csfUSR8w2u3ERzGUFnNgMbEh9hxnuKUUiF+FVP uBl7CTIJmq5wOX7kNUplAqP7dpG2o7xhExTWXcKW+TfAb036jJOQccZk9Q/N8cR8tGQ5vd kQFg+TgkBmMSQcRGb+ooLRnZNT1Vw/gktdd5DMrTVqmuD7JexTwW5EstS+fjbA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1683811978; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=GTa6a2eauZ6GG/IHWd8snoqDW1CfWJ+6oiXPFTyVHBo=; b=Dji3krBSTVGrCZG1sCq0MZ6okUjKY6E/9ot+4vFINQWk9mg7smeM5qqUoe7JE+2eVzkEaS 9yNDkAb/yFYqXefICxq8Oa4ifJCZJgBqal7V6EsLxkaLXCj+lyNSz+Nr5Ul2SFjd8sMi8v jdyvtXuY5bawpyIlzKJjpt2GyM55dLct99f3HTsA9LiT+++L/0a+BOkc/OiyN8fLJcLcQ8 vHbGLsGcEHXdB7aXCjjqg5VtrQ1e0S0Zd7MeKbIjkf0R3wZu8anWL89Jf7lZ+JFGenrlXg 3kc30+g5+0WIYQejPY3X6vdasyZZxvohcLeg+ovc16KJORtq8FSJitt2RcV6Cw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1683811978; a=rsa-sha256; cv=none; b=DDiIz7hA02u3TsEhbDOGtrzdej4QS3pJBFBmsiHYJpZxqs0B41X6mZBD3mlvFFTPGcXBMS Ru8GzRvwJjuCMqBNJKvC2ZrDoypRb6HRrE6qIYPL1l9LvuVe+2TsMi//PmCpGFFJzdAgIC 3BDCCpZF3yo+5xzGFCmmIeNB+FTbDxMbCQNhLmi+Gc+88Mt0QDfWJwv2icux4Lk/QVi66R HXYJqjZj/zfUOHA6f4sOwx+U+TBLzTR/CPSv9Xpirlpw09weWkKOC59YUIZIeJsJSTj6K9 nom+YbDuzpDhoVJlTBxzKZAZ6WRNIxAwV4TizCIsTqXaQ+RXAh7eeE773bFFWA== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4QHCWZ49z4zvHj; Thu, 11 May 2023 13:32:58 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 34BDWw8t076752; Thu, 11 May 2023 13:32:58 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 34BDWwwK076751; Thu, 11 May 2023 13:32:58 GMT (envelope-from git) Date: Thu, 11 May 2023 13:32:58 GMT Message-Id: <202305111332.34BDWwwK076751@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mark Johnston Subject: git: 0a300a95c10f - stable/13 - vmm: Dynamically allocate a couple of per-CPU state save areas List-Id: Commits to the stable branches of the FreeBSD src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-branches List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-branches@freebsd.org X-BeenThere: dev-commits-src-branches@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: markj X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 0a300a95c10f079d4735c9c13bd6bea90402a87a Auto-Submitted: auto-generated X-ThisMailContainsUnwantedMimeParts: N The branch stable/13 has been updated by markj: URL: https://cgit.FreeBSD.org/src/commit/?id=0a300a95c10f079d4735c9c13bd6bea90402a87a commit 0a300a95c10f079d4735c9c13bd6bea90402a87a Author: Mark Johnston AuthorDate: 2023-05-10 13:19:09 +0000 Commit: Mark Johnston CommitDate: 2023-05-11 13:32:32 +0000 vmm: Dynamically allocate a couple of per-CPU state save areas This avoids bloating the BSS when MAXCPU is large. No functional change intended. PR: 269572 Reviewed by: corvink, rew Tested by: rew MFC after: 2 weeks Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D39805 (cherry picked from commit 74ac712f72cfd6d7b3db3c9d3b72ccf2824aa183) --- sys/amd64/vmm/amd/svm.c | 11 ++++++----- sys/amd64/vmm/intel/vmx.c | 12 ++++++++---- 2 files changed, 14 insertions(+), 9 deletions(-) diff --git a/sys/amd64/vmm/amd/svm.c b/sys/amd64/vmm/amd/svm.c index d995b68ebdc6..0af7342128f2 100644 --- a/sys/amd64/vmm/amd/svm.c +++ b/sys/amd64/vmm/amd/svm.c @@ -43,6 +43,7 @@ __FBSDID("$FreeBSD$"); #include #include +#include #include #include @@ -123,10 +124,8 @@ SYSCTL_UINT(_hw_vmm_svm, OID_AUTO, num_asids, CTLFLAG_RDTUN, &nasid, 0, /* Current ASID generation for each host cpu */ static struct asid asid[MAXCPU]; -/* - * SVM host state saved area of size 4KB for each core. - */ -static uint8_t hsave[MAXCPU][PAGE_SIZE] __aligned(PAGE_SIZE); +/* SVM host state saved area of size 4KB for each physical core. */ +static uint8_t *hsave; static VMM_STAT_AMD(VCPU_EXITINTINFO, "VM exits during event delivery"); static VMM_STAT_AMD(VCPU_INTINFO_INJECTED, "Events pending at VM entry"); @@ -167,6 +166,7 @@ svm_modcleanup(void) { smp_rendezvous(NULL, svm_disable, NULL, NULL); + kmem_free(hsave, (mp_maxid + 1) * PAGE_SIZE); return (0); } @@ -214,7 +214,7 @@ svm_enable(void *arg __unused) efer |= EFER_SVM; wrmsr(MSR_EFER, efer); - wrmsr(MSR_VM_HSAVE_PA, vtophys(hsave[curcpu])); + wrmsr(MSR_VM_HSAVE_PA, vtophys(&hsave[curcpu * PAGE_SIZE])); } /* @@ -269,6 +269,7 @@ svm_modinit(int ipinum) svm_npt_init(ipinum); /* Enable SVM on all CPUs */ + hsave = kmem_malloc((mp_maxid + 1) * PAGE_SIZE, M_WAITOK | M_ZERO); smp_rendezvous(NULL, svm_enable, NULL, NULL); return (0); diff --git a/sys/amd64/vmm/intel/vmx.c b/sys/amd64/vmm/intel/vmx.c index 32e53de4e8ee..55ed5fdf1a00 100644 --- a/sys/amd64/vmm/intel/vmx.c +++ b/sys/amd64/vmm/intel/vmx.c @@ -46,6 +46,7 @@ __FBSDID("$FreeBSD$"); #include #include +#include #include #include @@ -134,7 +135,7 @@ SYSCTL_NODE(_hw_vmm, OID_AUTO, vmx, CTLFLAG_RW | CTLFLAG_MPSAFE, NULL, NULL); int vmxon_enabled[MAXCPU]; -static char vmxon_region[MAXCPU][PAGE_SIZE] __aligned(PAGE_SIZE); +static uint8_t *vmxon_region; static uint32_t pinbased_ctls, procbased_ctls, procbased_ctls2; static uint32_t exit_ctls, entry_ctls; @@ -618,6 +619,7 @@ vmx_modcleanup(void) nmi_flush_l1d_sw = 0; smp_rendezvous(NULL, vmx_disable, NULL, NULL); + kmem_free(vmxon_region, (mp_maxid + 1) * PAGE_SIZE); return (0); } @@ -638,8 +640,8 @@ vmx_enable(void *arg __unused) load_cr4(rcr4() | CR4_VMXE); - *(uint32_t *)vmxon_region[curcpu] = vmx_revision(); - error = vmxon(vmxon_region[curcpu]); + *(uint32_t *)&vmxon_region[curcpu * PAGE_SIZE] = vmx_revision(); + error = vmxon(&vmxon_region[curcpu * PAGE_SIZE]); if (error == 0) vmxon_enabled[curcpu] = 1; } @@ -649,7 +651,7 @@ vmx_modresume(void) { if (vmxon_enabled[curcpu]) - vmxon(vmxon_region[curcpu]); + vmxon(&vmxon_region[curcpu * PAGE_SIZE]); } static int @@ -953,6 +955,8 @@ vmx_modinit(int ipinum) vmx_msr_init(); /* enable VMX operation */ + vmxon_region = kmem_malloc((mp_maxid + 1) * PAGE_SIZE, + M_WAITOK | M_ZERO); smp_rendezvous(NULL, vmx_enable, NULL, NULL); vmx_initialized = 1;