From nobody Wed Nov 23 08:00:16 2022 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4NHD7h2Bw7z4j1Vk; Wed, 23 Nov 2022 08:00:16 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4NHD7h1hhZz45Gr; Wed, 23 Nov 2022 08:00:16 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1669190416; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=xi2IhE2s203tXnSyeLm3ZBW/KP3hhReAwlYGsxUqRWA=; b=UIf/HMu7LOnATfB75ZMkL0fIsyw+0phz+CM+h4sYVRgEwY//7oqFJpmNaO0+EpKnIneS3T PMMUNfoSnG0Pop59SgN2G/Hoqz02DfdjJu8aI5MWglZVSQlDd6VOZRRKP8M3FaaPzZSK8i Eu7ZASY0T2tiVvfIVOHk8S1n+4OIbZe42DLMt9yOXX8RRvkBhBu+CMwZ4YCq9jx+PsA8y9 fgUVIatF45fPI4ST8Ze2Re8RRWIbKtX7azutk5j+Jj8IN4fAdGsnfE425ZoV3bJ4FeSpLN k7oKJzI05AlZM+2GyQIwS4hNcSTOGn9YfOcq30mDeotWcXJrxNOELySjzs/7hQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1669190416; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=xi2IhE2s203tXnSyeLm3ZBW/KP3hhReAwlYGsxUqRWA=; b=nnD+WCkIKsRuCGPc4402/mCWtz+t/cXhxfkwzQU6VvfmvKkpp/8HnLsP8lWY2NAOR6xwy9 D0PBy4pn7EBrZq3A+jX37p741G78+Hyo93WuyBunFQ6E/JSPD8n4KOoEw4WR38Gl2pceBk LyPhc32bmj5S4KeBMHuS5BbqJpe2KRJBPbOFObAuw5A1+PK/gYfNTOYQbML5UvERBaApxl wZzBs60Jn181bP0IHgGaWTlTUjpGpmKOJcKtTmYDKTLyLyuQo0GmZqe4FkbuwiI/POWm4B pLc0M5WhBgInA9/wYiKgzrdrwYYVXpCbhg9sGy2enWWV1V+koCmwzuwaVklLPQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1669190416; a=rsa-sha256; cv=none; b=ftVxiFMn0lTtFCn9lAkqOu8ZplCGyFVuCojAiExjSXMnD0rVpdJ7v5NBa9ZO7zKI9z7Ixp Viq64JixvZvWWdqOwiTkSFHMP+3JwV/2e3Nzs2onv35Hl7Kbnc8Z+3NCE77K8nV/weYYn1 BXyTEUxZfrLhAQuFPEENARBq/8XMfIyMDCN0EUEvAAInTBdnpgykLtKuLkNMdbsEwDTyTd 59Dbt84FdMGzSyuHzbC7SmWgDGlENiP8pZ6jiezObTi+QNYdAWlwOW0z5JRsSMWPv2oWUs 1nWKCSkastdVtLIYpV6jzpC8Jbv895TlGQAJmLZL9ncRXIYNc10uApFjAmT5Dw== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4NHD7h0kySzF8p; Wed, 23 Nov 2022 08:00:16 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 2AN80GEP068426; Wed, 23 Nov 2022 08:00:16 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 2AN80G58068419; Wed, 23 Nov 2022 08:00:16 GMT (envelope-from git) Date: Wed, 23 Nov 2022 08:00:16 GMT Message-Id: <202211230800.2AN80G58068419@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: =?utf-8?Q?Corvin=20K=C3=B6hne?= Subject: git: 7c326ab5bb9a - main - vmm: don't lock a mtx in the icr_low write handler List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-main@freebsd.org X-BeenThere: dev-commits-src-main@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: corvink X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 7c326ab5bb9aced8dcbc2465ac1c9ff8df2ba46b Auto-Submitted: auto-generated X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by corvink: URL: https://cgit.FreeBSD.org/src/commit/?id=7c326ab5bb9aced8dcbc2465ac1c9ff8df2ba46b commit 7c326ab5bb9aced8dcbc2465ac1c9ff8df2ba46b Author: Corvin Köhne AuthorDate: 2022-11-21 14:00:04 +0000 Commit: Corvin Köhne CommitDate: 2022-11-23 08:00:04 +0000 vmm: don't lock a mtx in the icr_low write handler x2apic accesses are handled by a wrmsr exit. This handler is called in a critical section. So, we can't lock a mtx in the icr_low handler. Reported by: kp, pho Tested by: kp, pho Approved by: manu (mentor) Fixes: c0f35dbf19c3c8825bd2b321d8efd582807d1940 vmm: Use a cpuset_t for vCPUs waiting for STARTUP IPIs. MFC after: 1 week MFC with: c0f35dbf19c3c8825bd2b321d8efd582807d1940 Sponsored by: Beckhoff Automation GmbH & Co. KG Differential Revision: https://reviews.freebsd.org/D37452 --- sys/amd64/vmm/io/vlapic.c | 61 +++++++++++++++++++++++++++-------------------- 1 file changed, 35 insertions(+), 26 deletions(-) diff --git a/sys/amd64/vmm/io/vlapic.c b/sys/amd64/vmm/io/vlapic.c index 6307ce341c72..44641cc29035 100644 --- a/sys/amd64/vmm/io/vlapic.c +++ b/sys/amd64/vmm/io/vlapic.c @@ -1127,9 +1127,8 @@ vlapic_icrlo_write_handler(struct vlapic *vlapic, bool *retu) i == vlapic->vcpuid) break; - /* vCPU i is waiting for SIPI. */ - CPU_SETOF(i, &dmask); - vm_await_start(vlapic->vm, &dmask); + CPU_SETOF(i, &ipimask); + break; } @@ -1140,36 +1139,17 @@ vlapic_icrlo_write_handler(struct vlapic *vlapic, bool *retu) if (!phys) break; - /* - * Old bhyve versions don't support the IPI - * exit. Translate it into the old style. - */ i = vm_apicid2vcpuid(vlapic->vm, dest); if (i >= vm_get_maxcpus(vlapic->vm) || i == vlapic->vcpuid) break; - /* - * Ignore SIPIs in any state other than wait-for-SIPI - */ - CPU_SETOF(i, &dmask); - dmask = vm_start_cpus(vlapic->vm, &dmask); - if (CPU_EMPTY(&dmask)) - break; - - vmexit = vm_exitinfo(vlapic->vcpu); - vmexit->exitcode = VM_EXITCODE_SPINUP_AP; - vmexit->u.spinup_ap.vcpu = i; - vmexit->u.spinup_ap.rip = vec << PAGE_SHIFT; + CPU_SETOF(i, &ipimask); - *retu = true; break; } - /* - * Ignore SIPIs in any state other than wait-for-SIPI - */ - ipimask = vm_start_cpus(vlapic->vm, &dmask); + CPU_COPY(&dmask, &ipimask); break; default: return (1); @@ -1199,14 +1179,43 @@ vlapic_handle_init(struct vcpu *vcpu, void *arg) int vm_handle_ipi(struct vcpu *vcpu, struct vm_exit *vme, bool *retu) { + struct vlapic *vlapic = vm_lapic(vcpu); + cpuset_t *dmask = &vme->u.ipi.dmask; + uint8_t vec = vme->u.ipi.vector; + *retu = true; switch (vme->u.ipi.mode) { case APIC_DELMODE_INIT: - vm_smp_rendezvous(vcpu, vme->u.ipi.dmask, vlapic_handle_init, + vm_smp_rendezvous(vcpu, *dmask, vlapic_handle_init, NULL); - vm_await_start(vcpu_vm(vcpu), &vme->u.ipi.dmask); + vm_await_start(vcpu_vm(vcpu), dmask); + + if (!vlapic->ipi_exit) { + *retu = false; + } + break; case APIC_DELMODE_STARTUP: + /* + * Ignore SIPIs in any state other than wait-for-SIPI + */ + *dmask = vm_start_cpus(vcpu_vm(vcpu), dmask); + + if (CPU_EMPTY(dmask)) { + *retu = false; + break; + } + + /* + * Old bhyve versions don't support the IPI + * exit. Translate it into the old style. + */ + if (!vlapic->ipi_exit) { + vme->exitcode = VM_EXITCODE_SPINUP_AP; + vme->u.spinup_ap.vcpu = CPU_FFS(dmask); + vme->u.spinup_ap.rip = vec << PAGE_SHIFT; + } + break; default: return (1);