From nobody Fri Nov 18 21:24:14 2022 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4NDVCk2pW0z4hLRx; Fri, 18 Nov 2022 21:24:18 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from omta002.cacentral1.a.cloudfilter.net (omta002.cacentral1.a.cloudfilter.net [3.97.99.33]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "Client", Issuer "CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4NDVCk03g2z3vLC; Fri, 18 Nov 2022 21:24:17 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Authentication-Results: mx1.freebsd.org; none Received: from shw-obgw-4003a.ext.cloudfilter.net ([10.228.9.183]) by cmsmtp with ESMTP id w2iXosAtdyQ9ew8qPoVAT0; Fri, 18 Nov 2022 21:24:17 +0000 Received: from spqr.komquats.com ([70.66.148.124]) by cmsmtp with ESMTPA id w8qNoHqDA5QfLw8qOooLEU; Fri, 18 Nov 2022 21:24:17 +0000 X-Authority-Analysis: v=2.4 cv=SuCDVdC0 c=1 sm=1 tr=0 ts=6377f801 a=Cwc3rblV8FOMdVN/wOAqyQ==:117 a=Cwc3rblV8FOMdVN/wOAqyQ==:17 a=kj9zAlcOel0A:10 a=9xFQ1JgjjksA:10 a=6I5d2MoRAAAA:8 a=YxBL1-UpAAAA:8 a=EkcXrb_YAAAA:8 a=HCrfzB8IyKeE5FSjKyQA:9 a=CjuIK1q_8ugA:10 a=IjZwj45LgO3ly-622nXo:22 a=Ia-lj3WSrqcvXOmTRaiG:22 a=LK5xJRSDVpKd5WXXoEvA:22 Received: from slippy.cwsent.com (slippy [10.1.1.91]) by spqr.komquats.com (Postfix) with ESMTP id 0D91749D; Fri, 18 Nov 2022 13:24:15 -0800 (PST) Received: by slippy.cwsent.com (Postfix, from userid 1000) id E35931B2; Fri, 18 Nov 2022 13:24:14 -0800 (PST) X-Mailer: exmh version 2.9.0 11/07/2018 with nmh-1.7+dev Reply-to: Cy Schubert From: Cy Schubert X-os: FreeBSD X-Sender: cy@cwsent.com X-URL: http://www.cschubert.com/ To: FreeBSD User cc: John Baldwin , src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org Subject: Re: git: d8be3d523dd5 - main - vmm: Use struct vcpu in the rendezvous code. In-reply-to: <20221118193623.4dfe35bf@thor.intern.walstatt.dynvpn.de> References: <202211181826.2AIIQioj030460@gitrepo.freebsd.org> <20221118193623.4dfe35bf@thor.intern.walstatt.dynvpn.de> Comments: In-reply-to FreeBSD User message dated "Fri, 18 Nov 2022 19:35:56 +0100." List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-main@freebsd.org X-BeenThere: dev-commits-src-main@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Fri, 18 Nov 2022 13:24:14 -0800 Message-Id: <20221118212414.E35931B2@slippy.cwsent.com> X-CMAE-Envelope: MS4xfFLPMDSCOyqNIZlztP9y3jUFrHpV+TY59DNcMidUMlR+PhQdYNIQ3OEFFEjPW8Or5MyUXIp0xhN8Rn+8WBrHvRzvcMceT243kA+KagiUzuy2Lku1emIM ywjbcTZQdxTGT/0X24Q8J5AnVyRX8O38oMgoQi03C03ltHp+FT87nM7aGPm8vPxI7EMmDP2MI0OlLoN6RkRPzH1NrmHfwhjNzlA7A2jCjDBD9/ApuN6eZpuK uUAkzdTMb1fRHp5yv1y2RVAvyHazh/IFQvETAUXA9qyRnepkJFGlROOj18c09VmK7RxLGbAM0CokDBrvdxFemheMloIkI2jhIf0EuZxZmc4Rvf10Y+VVyD+F m9mB8GHN X-Rspamd-Queue-Id: 4NDVCk03g2z3vLC X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:16509, ipnet:3.96.0.0/15, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N In message <20221118193623.4dfe35bf@thor.intern.walstatt.dynvpn.de>, FreeBSD Us er writes: > Am Fri, 18 Nov 2022 18:26:44 GMT > John Baldwin schrieb: > > > The branch main has been updated by jhb: > > > > URL: https://cgit.FreeBSD.org/src/commit/?id=d8be3d523dd50a17f48957c1bb2e0c > d7bbf02cab > > > > commit d8be3d523dd50a17f48957c1bb2e0cd7bbf02cab > > Author: John Baldwin > > AuthorDate: 2022-11-18 18:03:34 +0000 > > Commit: John Baldwin > > CommitDate: 2022-11-18 18:25:37 +0000 > > > > vmm: Use struct vcpu in the rendezvous code. > > > > Reviewed by: corvink, markj > > Differential Revision: https://reviews.freebsd.org/D37165 > > --- > > sys/amd64/include/vmm.h | 4 ++-- > > sys/amd64/vmm/io/vioapic.c | 11 +++++------ > > sys/amd64/vmm/io/vlapic.c | 10 +++++----- > > sys/amd64/vmm/io/vlapic.h | 2 +- > > sys/amd64/vmm/vmm.c | 36 +++++++++++++++++------------------- > > 5 files changed, 30 insertions(+), 33 deletions(-) > > > > diff --git a/sys/amd64/include/vmm.h b/sys/amd64/include/vmm.h > > index 2f9a8776bb39..1c68181f5ff4 100644 > > --- a/sys/amd64/include/vmm.h > > +++ b/sys/amd64/include/vmm.h > > @@ -313,8 +313,8 @@ int vm_restore_time(struct vm *vm); > > * by 'dest' to be stalled. The caller should not rely on any vcpus making > > * forward progress when the rendezvous is in progress. > > */ > > -typedef void (*vm_rendezvous_func_t)(struct vm *vm, int vcpuid, void *arg) > ; > > -int vm_smp_rendezvous(struct vm *vm, int vcpuid, cpuset_t dest, > > +typedef void (*vm_rendezvous_func_t)(struct vcpu *vcpu, void *arg); > > +int vm_smp_rendezvous(struct vcpu *vcpu, cpuset_t dest, > > vm_rendezvous_func_t func, void *arg); > > cpuset_t vm_active_cpus(struct vm *vm); > > cpuset_t vm_debug_cpus(struct vm *vm); > > diff --git a/sys/amd64/vmm/io/vioapic.c b/sys/amd64/vmm/io/vioapic.c > > index 66a394af0d00..aee58849dd7d 100644 > > --- a/sys/amd64/vmm/io/vioapic.c > > +++ b/sys/amd64/vmm/io/vioapic.c > > @@ -237,7 +237,7 @@ vioapic_pulse_irq(struct vm *vm, int irq) > > * configuration. > > */ > > static void > > -vioapic_update_tmr(struct vm *vm, int vcpuid, void *arg) > > +vioapic_update_tmr(struct vcpu *vcpu, void *arg) > > { > > struct vioapic *vioapic; > > struct vlapic *vlapic; > > @@ -245,8 +245,8 @@ vioapic_update_tmr(struct vm *vm, int vcpuid, void *arg > ) > > int delmode, pin, vector; > > bool level, phys; > > > > - vlapic = vm_lapic(vm_vcpu(vm, vcpuid)); > > - vioapic = vm_ioapic(vm); > > + vlapic = vm_lapic(vcpu); > > + vioapic = vm_ioapic(vcpu_vm(vcpu)); > > > > VIOAPIC_LOCK(vioapic); > > /* > > @@ -317,10 +317,9 @@ vioapic_write(struct vioapic *vioapic, struct vcpu *vc > pu, uint32_t addr, > > { > > uint64_t data64, mask64; > > uint64_t last, changed; > > - int regnum, pin, lshift, vcpuid; > > + int regnum, pin, lshift; > > cpuset_t allvcpus; > > > > - vcpuid = vcpu_vcpuid(vcpu); > > regnum = addr & 0xff; > > switch (regnum) { > > case IOAPIC_ID: > > @@ -374,7 +373,7 @@ vioapic_write(struct vioapic *vioapic, struct vcpu *vcp > u, uint32_t addr, > > "vlapic trigger-mode register", pin); > > VIOAPIC_UNLOCK(vioapic); > > allvcpus = vm_active_cpus(vioapic->vm); > > - (void)vm_smp_rendezvous(vioapic->vm, vcpuid, allvcpus, > > + (void)vm_smp_rendezvous(vcpu, allvcpus, > > vioapic_update_tmr, NULL); > > VIOAPIC_LOCK(vioapic); > > } > > diff --git a/sys/amd64/vmm/io/vlapic.c b/sys/amd64/vmm/io/vlapic.c > > index 5075336e54bf..67a09401efb1 100644 > > --- a/sys/amd64/vmm/io/vlapic.c > > +++ b/sys/amd64/vmm/io/vlapic.c > > @@ -1200,9 +1200,9 @@ vlapic_icrlo_write_handler(struct vlapic *vlapic, boo > l *retu) > > } > > > > static void > > -vlapic_handle_init(struct vm *vm, int vcpuid, void *arg) > > +vlapic_handle_init(struct vcpu *vcpu, void *arg) > > { > > - struct vlapic *vlapic = vm_lapic(vm_vcpu(vm, vcpuid)); > > + struct vlapic *vlapic = vm_lapic(vcpu); > > > > vlapic_reset(vlapic); > > > > @@ -1211,13 +1211,13 @@ vlapic_handle_init(struct vm *vm, int vcpuid, void > *arg) > > } > > > > int > > -vm_handle_ipi(struct vm *vm, int vcpuid, struct vm_exit *vme, bool *retu) > > +vm_handle_ipi(struct vcpu *vcpu, struct vm_exit *vme, bool *retu) > > { > > *retu = true; > > switch (vme->u.ipi.mode) { > > case APIC_DELMODE_INIT: > > - vm_smp_rendezvous(vm, vcpuid, vme->u.ipi.dmask, > > - vlapic_handle_init, NULL); > > + vm_smp_rendezvous(vcpu, vme->u.ipi.dmask, vlapic_handle_init, > > + NULL); > > break; > > case APIC_DELMODE_STARTUP: > > break; > > diff --git a/sys/amd64/vmm/io/vlapic.h b/sys/amd64/vmm/io/vlapic.h > > index f8ac42fc7514..f6dd9493a5eb 100644 > > --- a/sys/amd64/vmm/io/vlapic.h > > +++ b/sys/amd64/vmm/io/vlapic.h > > @@ -115,6 +115,6 @@ void vlapic_self_ipi_handler(struct vlapic *vlapic, uin > t64_t val); > > int vlapic_snapshot(struct vm *vm, struct vm_snapshot_meta *meta); > > #endif > > > > -int vm_handle_ipi(struct vm *vm, int vcpuid, struct vm_exit *vme, bool *re > tu); > > +int vm_handle_ipi(struct vcpu *vcpu, struct vm_exit *vme, bool *retu); > > > > #endif /* _VLAPIC_H_ */ > > diff --git a/sys/amd64/vmm/vmm.c b/sys/amd64/vmm/vmm.c > > index cb1a5b62e4f4..f19076f27bb9 100644 > > --- a/sys/amd64/vmm/vmm.c > > +++ b/sys/amd64/vmm/vmm.c > > @@ -1320,15 +1320,14 @@ vcpu_require_state_locked(struct vm *vm, int vcpuid > , enum vcpu_state > > newstate) } > > > > static int > > -vm_handle_rendezvous(struct vm *vm, int vcpuid) > > +vm_handle_rendezvous(struct vcpu *vcpu) > > { > > + struct vm *vm = vcpu->vm; > > struct thread *td; > > - int error; > > - > > - KASSERT(vcpuid >= 0 && vcpuid < vm->maxcpus, > > - ("vm_handle_rendezvous: invalid vcpuid %d", vcpuid)); > > + int error, vcpuid; > > > > error = 0; > > + vcpuid = vcpu->vcpuid; > > td = curthread; > > mtx_lock(&vm->rendezvous_mtx); > > while (vm->rendezvous_func != NULL) { > > @@ -1337,18 +1336,18 @@ vm_handle_rendezvous(struct vm *vm, int vcpuid) > > > > if (CPU_ISSET(vcpuid, &vm->rendezvous_req_cpus) && > > !CPU_ISSET(vcpuid, &vm->rendezvous_done_cpus)) { > > - VCPU_CTR0(vm, vcpuid, "Calling rendezvous func"); > > - (*vm->rendezvous_func)(vm, vcpuid, vm->rendezvous_arg); > > + VMM_CTR0(vcpu, "Calling rendezvous func"); > > + (*vm->rendezvous_func)(vcpu, vm->rendezvous_arg); > > CPU_SET(vcpuid, &vm->rendezvous_done_cpus); > > } > > if (CPU_CMP(&vm->rendezvous_req_cpus, > > &vm->rendezvous_done_cpus) == 0) { > > - VCPU_CTR0(vm, vcpuid, "Rendezvous completed"); > > + VMM_CTR0(vcpu, "Rendezvous completed"); > > vm->rendezvous_func = NULL; > > wakeup(&vm->rendezvous_func); > > break; > > } > > - VCPU_CTR0(vm, vcpuid, "Wait for rendezvous completion"); > > + VMM_CTR0(vcpu, "Wait for rendezvous completion"); > > mtx_sleep(&vm->rendezvous_func, &vm->rendezvous_mtx, 0, > > "vmrndv", hz); > > if (td_ast_pending(td, TDA_SUSPEND)) { > > @@ -1626,7 +1625,7 @@ vm_handle_suspend(struct vm *vm, int vcpuid, bool *re > tu) > > } else { > > VCPU_CTR0(vm, vcpuid, "Rendezvous during suspend"); > > vcpu_unlock(vcpu); > > - error = vm_handle_rendezvous(vm, vcpuid); > > + error = vm_handle_rendezvous(vcpu); > > vcpu_lock(vcpu); > > } > > } > > @@ -1818,7 +1817,7 @@ restart: > > vme->u.ioapic_eoi.vector); > > break; > > case VM_EXITCODE_RENDEZVOUS: > > - error = vm_handle_rendezvous(vm, vcpuid); > > + error = vm_handle_rendezvous(vcpu); > > break; > > case VM_EXITCODE_HLT: > > intr_disabled = ((vme->u.hlt.rflags & PSL_I) == 0); > > @@ -1851,7 +1850,7 @@ restart: > > */ > > if (error == 0 && vme->exitcode == VM_EXITCODE_IPI) { > > retu = false; > > - error = vm_handle_ipi(vm, vcpuid, vme, &retu); > > + error = vm_handle_ipi(vcpu, vme, &retu); > > } > > > > if (error == 0 && retu == false) > > @@ -2564,17 +2563,16 @@ vm_apicid2vcpuid(struct vm *vm, int apicid) > > } > > > > int > > -vm_smp_rendezvous(struct vm *vm, int vcpuid, cpuset_t dest, > > +vm_smp_rendezvous(struct vcpu *vcpu, cpuset_t dest, > > vm_rendezvous_func_t func, void *arg) > > { > > + struct vm *vm = vcpu->vm; > > int error, i; > > > > /* > > * Enforce that this function is called without any locks > > */ > > WITNESS_WARN(WARN_PANIC, NULL, "vm_smp_rendezvous"); > > - KASSERT(vcpuid >= 0 && vcpuid < vm->maxcpus, > > - ("vm_smp_rendezvous: invalid vcpuid %d", vcpuid)); > > > > restart: > > mtx_lock(&vm->rendezvous_mtx); > > @@ -2584,9 +2582,9 @@ restart: > > * call the rendezvous handler in case this 'vcpuid' is one > > * of the targets of the rendezvous. > > */ > > - VCPU_CTR0(vm, vcpuid, "Rendezvous already in progress"); > > + VMM_CTR0(vcpu, "Rendezvous already in progress"); > > mtx_unlock(&vm->rendezvous_mtx); > > - error = vm_handle_rendezvous(vm, vcpuid); > > + error = vm_handle_rendezvous(vcpu); > > if (error != 0) > > return (error); > > goto restart; > > @@ -2594,7 +2592,7 @@ restart: > > KASSERT(vm->rendezvous_func == NULL, ("vm_smp_rendezvous: previous " > > "rendezvous is still in progress")); > > > > - VCPU_CTR0(vm, vcpuid, "Initiating rendezvous"); > > + VMM_CTR0(vcpu, "Initiating rendezvous"); > > vm->rendezvous_req_cpus = dest; > > CPU_ZERO(&vm->rendezvous_done_cpus); > > vm->rendezvous_arg = arg; > > @@ -2610,7 +2608,7 @@ restart: > > vcpu_notify_event(vm, i, false); > > } > > > > - return (vm_handle_rendezvous(vm, vcpuid)); > > + return (vm_handle_rendezvous(vcpu)); > > } > > > > struct vatpic * > > > > Buildworld fails on this commit: > > [...] > /usr/obj/usr/src/amd64.amd64/tmp/usr/include/machine/vmm.h:798:21: warning: d > eclaration of > 'struct vcpu' will not be visible outside of this function [-Wvisibility] vm_ > inject_ud(struct > vcpu *vcpu) ^ > /usr/obj/usr/src/amd64.amd64/tmp/usr/include/machine/vmm.h:800:36: error: too > few arguments to > function call, expected 5, have 4 vm_inject_fault(vcpu, IDT_UD, 0, 0); > ~~~~~~~~~~~~~~~ ^ > /usr/obj/usr/src/amd64.amd64/tmp/usr/include/machine/vmm.h:794:6: note: 'vm_i > nject_fault' > declared here void vm_inject_fault(void *vm, int vcpuid, int vector, int errc > ode_valid, > ^ > /usr/obj/usr/src/amd64.amd64/tmp/usr/include/machine/vmm.h:804:21: warning: d > eclaration of > 'struct vcpu' will not be visible outside of this function [-Wvisibility] vm_ > inject_gp(struct > vcpu *vcpu) ^ > /usr/obj/usr/src/amd64.amd64/tmp/usr/include/machine/vmm.h:806:36: error: too > few arguments to > function call, expected 5, have 4 vm_inject_fault(vcpu, IDT_GP, 1, 0); Sorry about that. This has been fixed. -- Cheers, Cy Schubert FreeBSD UNIX: Web: https://FreeBSD.org NTP: Web: https://nwtime.org e^(i*pi)+1=0