From nobody Tue Jan 18 09:19:18 2022 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 8588E1971E62; Tue, 18 Jan 2022 09:19:18 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4JdNWV3LfXz4clv; Tue, 18 Jan 2022 09:19:18 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1642497558; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=ycwmIhuBEGinRsuHxVdyqfnqYwOj+SlhFm9SENqTSPY=; b=SusjbbmAaoSBFFueaTLsyEjM78IAdIEKqn0bT9zKO9i+vrEaj5eko7zIgqo1sO4VoXp486 kw0GGieN5xmQhekYRKFelpLIKjOsfawW1JNkOGLxdEeMR2WQ+CWuEU/d8Ebnp46ASBPdUJ hP9eK6tuYf4ozQ6iUARn93+cV7CGK77z7zbiYp/ahdzVABo5DbEMkvfSixBksAIQcRob+S 88YslRU/yFBBqgvinioy4a02hgAahsfmyZHLxTCAzip4A3JCcMDZSXIYRGVhL2FLcL8ETQ j/mvZeC3MVvQa5X7wm3pyEDfLlI/b+Q5ppVLipMHCtOukbDMI0QSKcKk0iIYAQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 507F81E77C; Tue, 18 Jan 2022 09:19:18 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 20I9JIMb069745; Tue, 18 Jan 2022 09:19:18 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 20I9JIT6069744; Tue, 18 Jan 2022 09:19:18 GMT (envelope-from git) Date: Tue, 18 Jan 2022 09:19:18 GMT Message-Id: <202201180919.20I9JIT6069744@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: =?utf-8?Q?Roger Pau Monn=C3=A9?= Subject: git: 2450da67764c - main - x86/xen: use x{2}APIC if virtualized by hardware List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-main@freebsd.org X-BeenThere: dev-commits-src-main@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: royger X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 2450da67764c975868167db49106fb335af19f0d Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1642497558; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=ycwmIhuBEGinRsuHxVdyqfnqYwOj+SlhFm9SENqTSPY=; b=n3vQOiB8bHP+sT9+qjIZ+AyLU2jwRqH6zmdA7qfPrca2vgYIY0EmkJDHkl5cT2fiTwrsoi gFWwsBHEmE6rV4UVXHrg/XSgX5/KojNUMx+E2luNBXQivfhfiSssorKrvE7EQMfYdZqbpD JDTYktXYo3IsfifRv3iQev35iCCHkV+PR6lRTWsWGy5NNg0376Pc4Dozm7T0lroKX8PmtS HH6cKDzgf0p7aSRyOfD9Hw6X0nmVX2OGqeqExU6pUgda+gT7CHbRzBg+OWt5gYnkIit2PE yqLZG3RyOIJJ4jEV8uE8CXia25DyKBtZIqRWbD2ZnLCUgqtvnunpAYO6jbp9xQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1642497558; a=rsa-sha256; cv=none; b=TJKWLbGjnwmQgA02fjZ5CMwgPtLsdqO5cL7IdFwc+tlDpJNYiVkih0Gg7KVVJoIItycc7K rO8zx9Kpuj/WqiHyxm2W4qzC8cPasMPPdIknoOZBh1+TCRCiuniC9hV8f6e1xzhJ2St0YE Pu7EFhcdMV97DHi5irKaH8C9Fu4A/Xi91uzxd0WS4d5Rd3rN7JnmO/tmMDcyazIAW6SIe9 Kjh+Gpjh/MNESjOFId/EEF6NfqiJzXtvFEw2oEcn0O1/wel+TfhrUq/1R8enYjjFnC8z/x LFKntFj1KP8OqRTvbBZjzEy6LsZCY8wQa4v3BuvXA8UAj3M0rsgk0/z00+mxcg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by royger: URL: https://cgit.FreeBSD.org/src/commit/?id=2450da67764c975868167db49106fb335af19f0d commit 2450da67764c975868167db49106fb335af19f0d Author: Roger Pau Monné AuthorDate: 2022-01-13 13:57:07 +0000 Commit: Roger Pau Monné CommitDate: 2022-01-18 09:18:22 +0000 x86/xen: use x{2}APIC if virtualized by hardware Instead of using event channels or hypercalls to deal with IPIs and NMIs. Using a hardware virtualized APIC should be faster than using any PV interface, since the VM exit can be avoided. Xen exposes whether the domain is using hardware assisted x{2}APIC emulation in a CPUID bit. Sponsored by: Citrix Systems R&D --- sys/x86/include/xen/xen-os.h | 2 ++ sys/x86/xen/hvm.c | 16 ++++++++-------- sys/x86/xen/xen_apic.c | 13 +++++++++++++ 3 files changed, 23 insertions(+), 8 deletions(-) diff --git a/sys/x86/include/xen/xen-os.h b/sys/x86/include/xen/xen-os.h index 655961da22fd..4d4aa64955ba 100644 --- a/sys/x86/include/xen/xen-os.h +++ b/sys/x86/include/xen/xen-os.h @@ -49,6 +49,8 @@ extern int xen_disable_pv_disks; /* tunable for disabling PV nics */ extern int xen_disable_pv_nics; +extern uint32_t xen_cpuid_base; + static inline bool xen_has_percpu_evtchn(void) { diff --git a/sys/x86/xen/hvm.c b/sys/x86/xen/hvm.c index 569b113364b1..6eb16c3098fd 100644 --- a/sys/x86/xen/hvm.c +++ b/sys/x86/xen/hvm.c @@ -110,7 +110,7 @@ TUNABLE_INT("hw.xen.disable_pv_nics", &xen_disable_pv_nics); /*---------------------- XEN Hypervisor Probe and Setup ----------------------*/ -static uint32_t cpuid_base; +uint32_t xen_cpuid_base; static uint32_t xen_hvm_cpuid_base(void) @@ -153,7 +153,7 @@ hypervisor_version(void) uint32_t regs[4]; int major, minor; - do_cpuid(cpuid_base + 1, regs); + do_cpuid(xen_cpuid_base + 1, regs); major = regs[0] >> 16; minor = regs[0] & 0xffff; @@ -171,9 +171,9 @@ xen_hvm_init_hypercall_stubs(enum xen_hvm_init_type init_type) uint32_t regs[4]; /* Legacy PVH will get here without the cpuid leaf being set. */ - if (cpuid_base == 0) - cpuid_base = xen_hvm_cpuid_base(); - if (cpuid_base == 0) + if (xen_cpuid_base == 0) + xen_cpuid_base = xen_hvm_cpuid_base(); + if (xen_cpuid_base == 0) return (ENXIO); if (xen_domain() && init_type == XEN_HVM_INIT_LATE) { @@ -192,7 +192,7 @@ xen_hvm_init_hypercall_stubs(enum xen_hvm_init_type init_type) /* * Find the hypercall pages. */ - do_cpuid(cpuid_base + 2, regs); + do_cpuid(xen_cpuid_base + 2, regs); if (regs[0] != 1) return (EINVAL); @@ -448,8 +448,8 @@ xen_hvm_cpu_init(void) * Set vCPU ID. If available fetch the ID from CPUID, if not just use * the ACPI ID. */ - KASSERT(cpuid_base != 0, ("Invalid base Xen CPUID leaf")); - cpuid_count(cpuid_base + 4, 0, regs); + KASSERT(xen_cpuid_base != 0, ("Invalid base Xen CPUID leaf")); + cpuid_count(xen_cpuid_base + 4, 0, regs); KASSERT((regs[0] & XEN_HVM_CPUID_VCPU_ID_PRESENT) || !xen_pv_domain(), ("Xen PV domain without vcpu_id in cpuid")); diff --git a/sys/x86/xen/xen_apic.c b/sys/x86/xen/xen_apic.c index 2efa5a3c2345..c268e747f44a 100644 --- a/sys/x86/xen/xen_apic.c +++ b/sys/x86/xen/xen_apic.c @@ -54,6 +54,7 @@ __FBSDID("$FreeBSD$"); #include #include +#include #include /*--------------------------------- Macros -----------------------------------*/ @@ -601,11 +602,23 @@ xen_cpu_ipi_init(int cpu) static void xen_setup_cpus(void) { + uint32_t regs[4]; int i; if (!xen_vector_callback_enabled) return; + /* + * Check whether the APIC virtualization is hardware assisted, as + * that's faster than using event channels because it avoids the VM + * exit. + */ + KASSERT(xen_cpuid_base != 0, ("Invalid base Xen CPUID leaf")); + cpuid_count(xen_cpuid_base + 4, 0, regs); + if ((x2apic_mode && (regs[0] & XEN_HVM_CPUID_X2APIC_VIRT)) || + (!x2apic_mode && (regs[0] & XEN_HVM_CPUID_APIC_ACCESS_VIRT))) + return; + CPU_FOREACH(i) xen_cpu_ipi_init(i);