From nobody Fri Apr 28 10:40:22 2023 X-Original-To: dev-commits-src-branches@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q78JR0mC6z48JvN; Fri, 28 Apr 2023 10:40:23 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q78JR0Dydz3H8d; Fri, 28 Apr 2023 10:40:23 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1682678423; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=yxhPHWpPT5H+dXzS8ELHL2iYaRdFf/kZ9rwrE/Wz+D0=; b=FnFQdk889CVFj3c+jd+KpFrczhpkrDI5tLdwVI2R+nYCP6wyB3pzxmPkkyvxtRezpDFIF+ kV1P/cTk2udPeSZxSJ1CJ22JaJKQJyPKptUjEubBV1SpiJyZCplDqcg7XGbBPELHuvJXuG NkIiaQdfpSwyVOJf9A3LwFp+7rYvSORzZcvoNbgY2IseJnOs940xDAVy2fhcXFMDl02ZNn 8EMACTobMa5rCAN1ul10cGbQpoMg6LAnGlwVoPSB4FnFdK2gixg30uoSTVOW54U9zUJnhV loCGMYaBwlKnyi3waISdhiQ0tE/9SZaDvx/AhBxE1+39teOSKPdg/3eIOD/tSA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1682678423; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=yxhPHWpPT5H+dXzS8ELHL2iYaRdFf/kZ9rwrE/Wz+D0=; b=mp/yI+OnP+/YsepZCRV1lsevSgluRz6QGG0nG+RKSD3JSKFYQUprBy9tyHqZS2xt3Cxlg3 3MocvfnCAupCLsd8bdvBcXma5xuXnQz1djixIiw+5pNWIUb273LpjvvP1jbyoBWp5GRPHJ LstcE86KMVWAxXGXQkbXsjGvQ7jU1AWDb9faqT75FdB61WpBgsEWjbrEQluuDS+DjnX1gN bnuUOyDPiik7fEZYYR0GXfz08ohfNGNRwYz1uH3BNHYvcPXBLAYTRR+WDtKKpgIRE/fWzR nJ7Iudqf1f6fa2grVtIFIyC/J4hyAueTyeJu7hDKso8Yjp0oT5xHXL08mNGrsg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1682678423; a=rsa-sha256; cv=none; b=gQfrcCXfGNZjpdRjxbSx21jacIBSpbl/NvRT+ajhirvmla5B06d5LuHQi9kbxQwsdd7KVk eEUHv7laDydwVGXzIb7tci4kNdp8v5c7DhgSDsAzVJ9zzHflE31R8ukYlsyLGyqiBn9IGJ jpp0hbjju728Ly+KfOGsXc+Fe9BQO+5o+dVktuDSqD9MbgalwvCd1BZc7xJu15Y7H2KARj 06oHJZyENJGeJfTVFynDbvRzIaMd3UebTM4B9E9w15PqSE9TyWrCCXRSjuwb62ZTCDc/6g hYhUUnccxVqyIvyft/vQmpn+EcjgEInJYop/O/qo16/JeBMUognC0y2nNMPANQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4Q78JQ6D9Bzwrf; Fri, 28 Apr 2023 10:40:22 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 33SAeMZv092607; Fri, 28 Apr 2023 10:40:22 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 33SAeM15092606; Fri, 28 Apr 2023 10:40:22 GMT (envelope-from git) Date: Fri, 28 Apr 2023 10:40:22 GMT Message-Id: <202304281040.33SAeM15092606@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: =?utf-8?Q?Corvin=20K=C3=B6hne?= Subject: git: 847adbb116fb - stable/13 - bhyve: define array to protect passthru regs List-Id: Commits to the stable branches of the FreeBSD src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-branches List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-branches@freebsd.org X-BeenThere: dev-commits-src-branches@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: corvink X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 847adbb116fbc1de0c44ff57ab3b2a843c5f3c94 Auto-Submitted: auto-generated X-ThisMailContainsUnwantedMimeParts: N The branch stable/13 has been updated by corvink: URL: https://cgit.FreeBSD.org/src/commit/?id=847adbb116fbc1de0c44ff57ab3b2a843c5f3c94 commit 847adbb116fbc1de0c44ff57ab3b2a843c5f3c94 Author: Corvin Köhne AuthorDate: 2021-03-19 12:48:34 +0000 Commit: Corvin Köhne CommitDate: 2023-04-28 07:28:34 +0000 bhyve: define array to protect passthru regs GPU passthrough requires a special handling of some PCI config register. Therefore, we need a flexible approach for implementing it. Adding an array of handler meets this condition. Start by using the default handler for all accesses to the PCI config space. In upcoming commits, we can start to split the default handler into several handler for each register that requires emulation. Reviewed by: markj MFC after: 1 week Sponsored by: Beckhoff Automation GmbH & Co. KG Differential Revision: https://reviews.freebsd.org/D39291 (cherry picked from commit 931bb7bf1c8f90f3303b5291e4868c587f9364e7) --- usr.sbin/bhyve/pci_passthru.c | 57 +++++++++++++++++++++++++++++++++++++------ usr.sbin/bhyve/pci_passthru.h | 9 +++++++ 2 files changed, 58 insertions(+), 8 deletions(-) diff --git a/usr.sbin/bhyve/pci_passthru.c b/usr.sbin/bhyve/pci_passthru.c index f42bbbda655f..79068ad9b6da 100644 --- a/usr.sbin/bhyve/pci_passthru.c +++ b/usr.sbin/bhyve/pci_passthru.c @@ -93,6 +93,9 @@ struct passthru_softc { int capoff; } psc_msix; struct pcisel psc_sel; + + cfgread_handler psc_pcir_rhandler[PCI_REGMAX + 1]; + cfgwrite_handler psc_pcir_whandler[PCI_REGMAX + 1]; }; static int @@ -643,6 +646,23 @@ done: return (error); } +int +set_pcir_handler(struct passthru_softc *sc, int reg, int len, + cfgread_handler rhandler, cfgwrite_handler whandler) +{ + if (reg > PCI_REGMAX || reg + len > PCI_REGMAX + 1) + return (-1); + + for (int i = reg; i < reg + len; ++i) { + assert(sc->psc_pcir_rhandler[i] == NULL || rhandler == NULL); + assert(sc->psc_pcir_whandler[i] == NULL || whandler == NULL); + sc->psc_pcir_rhandler[i] = rhandler; + sc->psc_pcir_whandler[i] = whandler; + } + + return (0); +} + static int passthru_legacy_config(nvlist_t *nvl, const char *opts) { @@ -902,12 +922,9 @@ msixcap_access(struct passthru_softc *sc, int coff) } static int -passthru_cfgread(struct pci_devinst *pi, int coff, int bytes, uint32_t *rv) +passthru_cfgread_default(struct passthru_softc *sc, + struct pci_devinst *pi __unused, int coff, int bytes, uint32_t *rv) { - struct passthru_softc *sc; - - sc = pi->pi_arg; - /* * PCI BARs and MSI capability is emulated. */ @@ -946,14 +963,25 @@ passthru_cfgread(struct pci_devinst *pi, int coff, int bytes, uint32_t *rv) } static int -passthru_cfgwrite(struct pci_devinst *pi, int coff, int bytes, uint32_t val) +passthru_cfgread(struct pci_devinst *pi, int coff, int bytes, uint32_t *rv) { - int error, msix_table_entries, i; struct passthru_softc *sc; - uint16_t cmd_old; sc = pi->pi_arg; + if (sc->psc_pcir_rhandler[coff] != NULL) + return (sc->psc_pcir_rhandler[coff](sc, pi, coff, bytes, rv)); + + return (passthru_cfgread_default(sc, pi, coff, bytes, rv)); +} + +static int +passthru_cfgwrite_default(struct passthru_softc *sc, struct pci_devinst *pi, + int coff, int bytes, uint32_t val) +{ + int error, msix_table_entries, i; + uint16_t cmd_old; + /* * PCI BARs are emulated */ @@ -1026,6 +1054,19 @@ passthru_cfgwrite(struct pci_devinst *pi, int coff, int bytes, uint32_t val) return (0); } +static int +passthru_cfgwrite(struct pci_devinst *pi, int coff, int bytes, uint32_t val) +{ + struct passthru_softc *sc; + + sc = pi->pi_arg; + + if (sc->psc_pcir_whandler[coff] != NULL) + return (sc->psc_pcir_whandler[coff](sc, pi, coff, bytes, val)); + + return (passthru_cfgwrite_default(sc, pi, coff, bytes, val)); +} + static void passthru_write(struct pci_devinst *pi, int baridx, uint64_t offset, int size, uint64_t value) diff --git a/usr.sbin/bhyve/pci_passthru.h b/usr.sbin/bhyve/pci_passthru.h index bdc8dc3cd0e2..f6ce2d4b85ab 100644 --- a/usr.sbin/bhyve/pci_passthru.h +++ b/usr.sbin/bhyve/pci_passthru.h @@ -11,5 +11,14 @@ #include "pci_emul.h" +struct passthru_softc; + +typedef int (*cfgread_handler)(struct passthru_softc *sc, + struct pci_devinst *pi, int coff, int bytes, uint32_t *rv); +typedef int (*cfgwrite_handler)(struct passthru_softc *sc, + struct pci_devinst *pi, int coff, int bytes, uint32_t val); + uint32_t read_config(const struct pcisel *sel, long reg, int width); void write_config(const struct pcisel *sel, long reg, int width, uint32_t data); +int set_pcir_handler(struct passthru_softc *sc, int reg, int len, + cfgread_handler rhandler, cfgwrite_handler whandler);