From nobody Thu Jan 26 22:12:13 2023 X-Original-To: dev-commits-src-branches@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4P2w194mlrz3c8kl; Thu, 26 Jan 2023 22:12:13 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4P2w1927fRz3wls; Thu, 26 Jan 2023 22:12:13 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1674771133; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=TIgRVQY3q9uP5HDGzteSSjQtLRz7b2YzJQ2Funxsaxg=; b=b0tfQxCH/pUS6iV43PQRY3/JYUZa/b0WSl/TA6pcIwqrhAB960dJPrWcBfmEVcu+5MHVAe fZQhxgLIqWdyhz9CIf8bmXtSxGvGmfRe5bFFwI2fOkA8XOF/3HMjEDQBWBjBRcJEwRcIiu oZq8A3vQTh7okyDfgW0CMDU5Dq3/XodI5cYtV2YokwcRI4lO+gLVpbJTZIdCQd60N67Cdp emsV6vsVJr/ySs5PzSYyChbPXVnkm6Dhj/+vB8PtHl0gYf43R6kmpNhmjWiNQFitVFGu4W OxF1ryg4SDYjNbkiVWjpE6KYYedGNUCdfFeN/oU6eGs433SpF4+z7legFoDSnA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1674771133; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=TIgRVQY3q9uP5HDGzteSSjQtLRz7b2YzJQ2Funxsaxg=; b=s/wdZjzkSMfZf34CVlGCYK3eXrOIs8H4Ar5nB4lIeklt/7p95GJi5Uaz57SvlouA1pYaD3 vqvUjkgtnUjJDKy3a87XPBVLDRTdZ/KPk36VWAYcRGR2DZcJizlWisK758q9ZVw4Hlt6DO eRBVKDm+xLcSjKdH9EDhbHbIhAoM47qGbtFjNDlTlzD6LjU79piFx1rXpUTePPhYSo9VEB SH1DMC9Rwd5T85kvx9bOYDHeLV6isZvCOP8Fm90SBX6Jz6cmyPJCKod6zsoTrwnv5XADzv WaRPUWoYyrW3qkHVrnCptRW/yD1DNaERvND6Mqor7KHoXXx7NGbrOTah3mZTFA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1674771133; a=rsa-sha256; cv=none; b=Zy0J6FG+bn4tQgY7KxYHjT4crL0TchGiPhP3imag1LwAQ/Fy8d8iYrIaOj9jox6lIkh7m+ 1N/p/v+M8s9Bw1nJskmeYm0SzF9CWgwfh0A8WW93AX3v2d6KMdUJq7p8BMJZpf/mxgdaqV 5KW/gokC+X6r9KdHdUr58OTorsRuZUMM6fSCnI/yY0dlY8az4H9xrWZt3Q313WeX8CT8sS slPrd9WZSxz5czPWGNkjJ2QmZbWOt+FKLBqvK38OZcyAtLuzdFCirm5scdeDBtplr2mzPO N9WOfQoKlirnlMac83icn5V9ATKapGf35h1tYcj8pIIsywAx2EHdWaYLmZozSw== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4P2w191BKVzly3; Thu, 26 Jan 2023 22:12:13 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 30QMCDZR022386; Thu, 26 Jan 2023 22:12:13 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 30QMCDX2022385; Thu, 26 Jan 2023 22:12:13 GMT (envelope-from git) Date: Thu, 26 Jan 2023 22:12:13 GMT Message-Id: <202301262212.30QMCDX2022385@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: John Baldwin Subject: git: 5d3ee23b91f3 - stable/13 - vmm: don't lock a mtx in the icr_low write handler List-Id: Commits to the stable branches of the FreeBSD src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-branches List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-branches@freebsd.org X-BeenThere: dev-commits-src-branches@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: jhb X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 5d3ee23b91f354f6dea2707cdf4a0fb813488106 Auto-Submitted: auto-generated X-ThisMailContainsUnwantedMimeParts: N The branch stable/13 has been updated by jhb: URL: https://cgit.FreeBSD.org/src/commit/?id=5d3ee23b91f354f6dea2707cdf4a0fb813488106 commit 5d3ee23b91f354f6dea2707cdf4a0fb813488106 Author: Corvin Köhne AuthorDate: 2022-11-21 14:00:04 +0000 Commit: John Baldwin CommitDate: 2023-01-26 22:06:03 +0000 vmm: don't lock a mtx in the icr_low write handler x2apic accesses are handled by a wrmsr exit. This handler is called in a critical section. So, we can't lock a mtx in the icr_low handler. Reported by: kp, pho Tested by: kp, pho Approved by: manu (mentor) Fixes: c0f35dbf19c3c8825bd2b321d8efd582807d1940 vmm: Use a cpuset_t for vCPUs waiting for STARTUP IPIs. MFC after: 1 week MFC with: c0f35dbf19c3c8825bd2b321d8efd582807d1940 Sponsored by: Beckhoff Automation GmbH & Co. KG Differential Revision: https://reviews.freebsd.org/D37452 (cherry picked from commit 7c326ab5bb9aced8dcbc2465ac1c9ff8df2ba46b) --- sys/amd64/vmm/io/vlapic.c | 61 +++++++++++++++++++++++++++-------------------- 1 file changed, 35 insertions(+), 26 deletions(-) diff --git a/sys/amd64/vmm/io/vlapic.c b/sys/amd64/vmm/io/vlapic.c index e13cdcc63d57..fd716fcc0b61 100644 --- a/sys/amd64/vmm/io/vlapic.c +++ b/sys/amd64/vmm/io/vlapic.c @@ -1127,9 +1127,8 @@ vlapic_icrlo_write_handler(struct vlapic *vlapic, bool *retu) i == vlapic->vcpuid) break; - /* vCPU i is waiting for SIPI. */ - CPU_SETOF(i, &dmask); - vm_await_start(vlapic->vm, &dmask); + CPU_SETOF(i, &ipimask); + break; } @@ -1140,36 +1139,17 @@ vlapic_icrlo_write_handler(struct vlapic *vlapic, bool *retu) if (!phys) break; - /* - * Old bhyve versions don't support the IPI - * exit. Translate it into the old style. - */ i = vm_apicid2vcpuid(vlapic->vm, dest); if (i >= vm_get_maxcpus(vlapic->vm) || i == vlapic->vcpuid) break; - /* - * Ignore SIPIs in any state other than wait-for-SIPI - */ - CPU_SETOF(i, &dmask); - dmask = vm_start_cpus(vlapic->vm, &dmask); - if (CPU_EMPTY(&dmask)) - break; - - vmexit = vm_exitinfo(vlapic->vcpu); - vmexit->exitcode = VM_EXITCODE_SPINUP_AP; - vmexit->u.spinup_ap.vcpu = i; - vmexit->u.spinup_ap.rip = vec << PAGE_SHIFT; + CPU_SETOF(i, &ipimask); - *retu = true; break; } - /* - * Ignore SIPIs in any state other than wait-for-SIPI - */ - ipimask = vm_start_cpus(vlapic->vm, &dmask); + CPU_COPY(&dmask, &ipimask); break; default: return (1); @@ -1199,14 +1179,43 @@ vlapic_handle_init(struct vcpu *vcpu, void *arg) int vm_handle_ipi(struct vcpu *vcpu, struct vm_exit *vme, bool *retu) { + struct vlapic *vlapic = vm_lapic(vcpu); + cpuset_t *dmask = &vme->u.ipi.dmask; + uint8_t vec = vme->u.ipi.vector; + *retu = true; switch (vme->u.ipi.mode) { case APIC_DELMODE_INIT: - vm_smp_rendezvous(vcpu, vme->u.ipi.dmask, vlapic_handle_init, + vm_smp_rendezvous(vcpu, *dmask, vlapic_handle_init, NULL); - vm_await_start(vcpu_vm(vcpu), &vme->u.ipi.dmask); + vm_await_start(vcpu_vm(vcpu), dmask); + + if (!vlapic->ipi_exit) { + *retu = false; + } + break; case APIC_DELMODE_STARTUP: + /* + * Ignore SIPIs in any state other than wait-for-SIPI + */ + *dmask = vm_start_cpus(vcpu_vm(vcpu), dmask); + + if (CPU_EMPTY(dmask)) { + *retu = false; + break; + } + + /* + * Old bhyve versions don't support the IPI + * exit. Translate it into the old style. + */ + if (!vlapic->ipi_exit) { + vme->exitcode = VM_EXITCODE_SPINUP_AP; + vme->u.spinup_ap.vcpu = CPU_FFS(dmask); + vme->u.spinup_ap.rip = vec << PAGE_SHIFT; + } + break; default: return (1);