From nobody Fri Aug 19 22:25:33 2022 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4M8btQ18VGz4ZLZD; Fri, 19 Aug 2022 22:25:34 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4M8btQ0WK0z3pNg; Fri, 19 Aug 2022 22:25:34 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1660947934; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=irXFPkG3V6s02Ujg8t040ZVBLJikBEWn0vhfDFiqFE8=; b=xRiVsTz5w5C2NZHNp33xVDAtzQIhd6UTTkOxA1Aezt4FX2uSFvq+HWZDjuT4sVd99qFm+e CTZvZTkGAg+XyXAGTt4Vewl11Ma+WOQTzbkMgaf4chpdcy7xyddZZQcQSFq4ezuJy3MHRG o4O7avFThNK5YL1xBGvdNzEA6Owdovvvb6MyN+cxBdM8LTuMwgvRtTsUS16I+LbGgikA5q 7axs2vj8d9jDs8B/KWbfy7JDyf/MuXAc2VwkXqaRg0ptqAAfpdwjBjj+1Ry3BYthaPZa4o 81gk3Cgluocz8JxklepisOtBLPYyH+BL6JtI7nldYXI/nG1kgA1PjNsZCV4xiw== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4M8btP6gjGz17y3; Fri, 19 Aug 2022 22:25:33 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 27JMPXQ6068475; Fri, 19 Aug 2022 22:25:33 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 27JMPXtu068474; Fri, 19 Aug 2022 22:25:33 GMT (envelope-from git) Date: Fri, 19 Aug 2022 22:25:33 GMT Message-Id: <202208192225.27JMPXtu068474@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: John Baldwin Subject: git: baf753cc196e - main - bhyve: Support other schemes for naming pass-through devices. List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-main@freebsd.org X-BeenThere: dev-commits-src-main@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: jhb X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: baf753cc196e2308828c7fb9b0385d5fe17ecdc2 Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1660947934; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=irXFPkG3V6s02Ujg8t040ZVBLJikBEWn0vhfDFiqFE8=; b=tM0l91SrsSNXLM6u2DTKQ5EmRiXC1n2ozONV3QgJIzBoy1z2EX3yVFjuT3VaRh0oh/VQ7k BF7nGJxhqMiiqLtTSp2ySytk3RjyLjnzqay/tAjo2lHnHelz2xtiaZxf5fTixaTXDOpnV9 cMlouK5ToTXBafaV34QxN3a84I0eLlNtK62A1qIG+NTHgAPm6CInKWs+Chz4VQQwyByA9t kieEj2yRkSOWqO5ilpXoQwc8Y6gu5MNPguOxRpfBa5wExhn0H+Ncge1eS8PA6Xj1zC/VoN mAqqgiOzIYgR44tiK5dH9DLZsZ3G9Xv8XYcwPsH5k427oJjUM8CqOseNUJDquQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1660947934; a=rsa-sha256; cv=none; b=dYCc6aAOOz6XqRpDN8f4KFpha5spnY6C3EDKsuUP/718d2hJHYDcfvKv9gVG15i5gdBh/g Q1/IUD1hzgu05wCIu1jAwkM4Y5ZpF5tKifSfVO8tTmXzJos839r8oH08OwN9T79CSIxLp1 6Ae5Ib00kWGLtTL6LdGMoiQfc9PEjA863bs3lXCq6+j6BkHv6DQwbRqAjevaJq+Tu/924R nPzIw53Esy5p7oRpsnqyizG+hy0e/xwGkVcPqv9Z8nhHU0sKEP9odW7MgkMEg3ovtBTaEu vGPeRGRAhf6hPm3bBAe5YElkuYXKTmfCcdIS9SEEui81lP55NttJrn9vfXAd/Q== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by jhb: URL: https://cgit.FreeBSD.org/src/commit/?id=baf753cc196e2308828c7fb9b0385d5fe17ecdc2 commit baf753cc196e2308828c7fb9b0385d5fe17ecdc2 Author: John Baldwin AuthorDate: 2022-08-19 21:55:29 +0000 Commit: John Baldwin CommitDate: 2022-08-19 21:58:55 +0000 bhyve: Support other schemes for naming pass-through devices. Permit naming pass through devices using the syntax accepted by pciconf (pci[:]::) as well as by device name (e.g. "ppt0"). While here, fix an error in the manpage that had the bus and slot arguments for the original /-delimited scheme swapped. Reviewed by: imp, markj Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D36147 --- usr.sbin/bhyve/bhyve.8 | 27 ++++++++-- usr.sbin/bhyve/bhyve_config.5 | 12 ++++- usr.sbin/bhyve/pci_passthru.c | 112 ++++++++++++++++++++++++++++++++++++------ 3 files changed, 131 insertions(+), 20 deletions(-) diff --git a/usr.sbin/bhyve/bhyve.8 b/usr.sbin/bhyve/bhyve.8 index 92534d226499..84e031f1340c 100644 --- a/usr.sbin/bhyve/bhyve.8 +++ b/usr.sbin/bhyve/bhyve.8 @@ -24,7 +24,7 @@ .\" .\" $FreeBSD$ .\" -.Dd November 28, 2021 +.Dd August 19, 2022 .Dt BHYVE 8 .Os .Sh NAME @@ -390,6 +390,7 @@ Network device backends: .Xc .El .Sm on +.Pp If .Cm mac is not specified, the MAC address is derived from a fixed OUI and the @@ -520,14 +521,32 @@ to that file. .El .Pp Pass-through device backends: -.Bl -tag -width 10n -.It Ns Ar slot Ns Cm \&/ Ns Ar bus Ns Cm \&/ Ns Ar function -Connect to a PCI device on the host at the selector described by +.Sm off +.Bl -bullet +.It +.Cm ppt Ar N Oo , Ar passthru-device-options Oc +.It +.Ns Ar bus Cm \&/ Ar slot Cm \&/ Ar function +.Op , Ar passthru-device-options +.It +.Cm pci Ar bus Cm : Ar slot Cm : Ns Ar function +.Op , Ar passthru-device-options +.El +.Sm on +.Pp +Connect to a PCI device on the host either named ppt +.Ns Ar N +or at the selector described by .Ar slot , .Ar bus , and .Ar function numbers. +.Pp +The +.Ar passthru-device-options +are: +.Bl -tag -width 10n .It Cm rom= Ns Ar romfile Add .Ar romfile diff --git a/usr.sbin/bhyve/bhyve_config.5 b/usr.sbin/bhyve/bhyve_config.5 index 20109170b094..94a8a4d5cb1d 100644 --- a/usr.sbin/bhyve/bhyve_config.5 +++ b/usr.sbin/bhyve/bhyve_config.5 @@ -23,7 +23,7 @@ .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF .\" SUCH DAMAGE. .\" -.Dd June 30, 2022 +.Dd August 19, 2022 .Dt BHYVE_CONFIG 5 .Os .Sh NAME @@ -575,6 +575,12 @@ If set, allocate a memory disk as the backing store. The value of this variable is the size of the memory disk in megabytes. .El .Ss PCI Passthrough Settings +The +.Xr ppt 4 +device driver must be attached to the +PCI device being passed through. +The device to pass through can be identified either by name or its +host PCI bus location. .Bl -column "Name" "integer" "Default" .It Sy Name Ta Sy Format Ta Sy Default Ta Sy Description .It Va bus Ta integer Ta Ta @@ -583,6 +589,10 @@ Host PCI bus address of device to pass through. Host PCI slot address of device to pass through. .It Va func Ta integer Ta Ta Host PCI function address of device to pass through. +.It Va pptdev Ta string Ta Ta +Name of a +.Xr ppt 4 +device to pass through. .It Va rom Ta path Ta Ta ROM file of the device which will be executed by OVMF to init the device. .El diff --git a/usr.sbin/bhyve/pci_passthru.c b/usr.sbin/bhyve/pci_passthru.c index 2b2f240fafe3..43e70d42e2b2 100644 --- a/usr.sbin/bhyve/pci_passthru.c +++ b/usr.sbin/bhyve/pci_passthru.c @@ -52,6 +52,7 @@ __FBSDID("$FreeBSD$"); #ifndef WITHOUT_CAPSICUM #include #endif +#include #include #include #include @@ -132,7 +133,7 @@ pcifd_init(void) errx(EX_OSERR, "Unable to apply rights for sandbox"); const cap_ioctl_t pcifd_ioctls[] = { PCIOCREAD, PCIOCWRITE, PCIOCGETBAR, - PCIOCBARIO, PCIOCBARMMAP }; + PCIOCBARIO, PCIOCBARMMAP, PCIOCGETCONF }; if (caph_ioctls_limit(pcifd, pcifd_ioctls, nitems(pcifd_ioctls)) == -1) errx(EX_OSERR, "Unable to apply rights for sandbox"); #endif @@ -645,30 +646,39 @@ done: static int passthru_legacy_config(nvlist_t *nvl, const char *opts) { + const char *cp; + char *tofree; char value[16]; int bus, slot, func; if (opts == NULL) return (0); - if (sscanf(opts, "%d/%d/%d", &bus, &slot, &func) != 3) { + cp = strchr(opts, ','); + + if (strncmp(opts, "ppt", strlen("ppt")) == 0) { + tofree = strndup(opts, cp - opts); + set_config_value_node(nvl, "pptdev", tofree); + free(tofree); + } else if (sscanf(opts, "pci0:%d:%d:%d", &bus, &slot, &func) == 3 || + sscanf(opts, "pci%d:%d:%d", &bus, &slot, &func) == 3 || + sscanf(opts, "%d/%d/%d", &bus, &slot, &func) == 3) { + snprintf(value, sizeof(value), "%d", bus); + set_config_value_node(nvl, "bus", value); + snprintf(value, sizeof(value), "%d", slot); + set_config_value_node(nvl, "slot", value); + snprintf(value, sizeof(value), "%d", func); + set_config_value_node(nvl, "func", value); + } else { EPRINTLN("passthru: invalid options \"%s\"", opts); return (-1); } - snprintf(value, sizeof(value), "%d", bus); - set_config_value_node(nvl, "bus", value); - snprintf(value, sizeof(value), "%d", slot); - set_config_value_node(nvl, "slot", value); - snprintf(value, sizeof(value), "%d", func); - set_config_value_node(nvl, "func", value); - - opts = strchr(opts, ','); - if (opts == NULL) { + if (cp == NULL) { return (0); } - return pci_parse_legacy_config(nvl, opts + 1); + return (pci_parse_legacy_config(nvl, cp + 1)); } static int @@ -722,6 +732,72 @@ passthru_init_rom(struct vmctx *const ctx, struct passthru_softc *const sc, return (0); } +static bool +passthru_lookup_pptdev(const char *name, int *bus, int *slot, int *func) +{ + struct pci_conf_io pc; + struct pci_conf conf[1]; + struct pci_match_conf patterns[1]; + char *cp; + + bzero(&pc, sizeof(struct pci_conf_io)); + pc.match_buf_len = sizeof(conf); + pc.matches = conf; + + bzero(&patterns, sizeof(patterns)); + + /* + * The pattern structure requires the unit to be split out from + * the driver name. Walk backwards from the end of the name to + * find the start of the unit. + */ + cp = strchr(name, '\0'); + assert(cp != NULL); + while (cp != name && isdigit(cp[-1])) + cp--; + if (cp == name || !isdigit(*cp)) { + EPRINTLN("Invalid passthru device name %s", name); + return (false); + } + if ((size_t)(cp - name) + 1 > sizeof(patterns[0].pd_name)) { + EPRINTLN("Passthru device name %s is too long", name); + return (false); + } + memcpy(patterns[0].pd_name, name, cp - name); + patterns[0].pd_unit = strtol(cp, &cp, 10); + if (*cp != '\0') { + EPRINTLN("Invalid passthru device name %s", name); + return (false); + } + patterns[0].flags = PCI_GETCONF_MATCH_NAME | PCI_GETCONF_MATCH_UNIT; + pc.num_patterns = 1; + pc.pat_buf_len = sizeof(patterns); + pc.patterns = patterns; + + if (ioctl(pcifd, PCIOCGETCONF, &pc) == -1) { + EPRINTLN("ioctl(PCIOCGETCONF): %s", strerror(errno)); + return (false); + } + if (pc.status != PCI_GETCONF_LAST_DEVICE && + pc.status != PCI_GETCONF_MORE_DEVS) { + EPRINTLN("error returned from PCIOCGETCONF ioctl"); + return (false); + } + if (pc.num_matches == 0) { + EPRINTLN("Passthru device %s not found", name); + return (false); + } + + if (conf[0].pc_sel.pc_domain != 0) { + EPRINTLN("Passthru device %s on unsupported domain", name); + return (false); + } + *bus = conf[0].pc_sel.pc_bus; + *slot = conf[0].pc_sel.pc_dev; + *func = conf[0].pc_sel.pc_func; + return (true); +} + static int passthru_init(struct vmctx *ctx, struct pci_devinst *pi, nvlist_t *nvl) { @@ -751,9 +827,15 @@ passthru_init(struct vmctx *ctx, struct pci_devinst *pi, nvlist_t *nvl) var = atoi(value); \ } while (0) - GET_INT_CONFIG(bus, "bus"); - GET_INT_CONFIG(slot, "slot"); - GET_INT_CONFIG(func, "func"); + value = get_config_value_node(nvl, "pptdev"); + if (value != NULL) { + if (!passthru_lookup_pptdev(value, &bus, &slot, &func)) + return (error); + } else { + GET_INT_CONFIG(bus, "bus"); + GET_INT_CONFIG(slot, "slot"); + GET_INT_CONFIG(func, "func"); + } if (vm_assign_pptdev(ctx, bus, slot, func) != 0) { warnx("PCI device at %d/%d/%d is not using the ppt(4) driver",