From nobody Fri Apr 28 10:40:23 2023 X-Original-To: dev-commits-src-branches@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q78JS2Fh1z48JvQ; Fri, 28 Apr 2023 10:40:24 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q78JS1Y2Jz3HF7; Fri, 28 Apr 2023 10:40:24 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1682678424; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=FTss0UYWKma8Gaxrt5W8VgWbf1W7v05cjnJYCttJh08=; b=pEPK3o2XhO/9t10ZJT9ttRVemxas16k3KWC5AclPs4ipToL44R5jm6S3Rk2o7M5GTpwBpU uwvh8vPcK8SL19IumL+vc7bxtEfrE0Y3ptkAtHyylQMF9yacov1DwP8Vk+CeiSBKbj+FJH nEPtTjaaI75K27rksiwEJCbks7wGtWIR5uqrtOw7Gog22iRXQ0fmgEcmXKlbqB54udHB/0 lggvBxcvGvsmnr1m1x3z+vyPp1qq09zktcrtRmXb16eUrCgKtNrlKq6YI69/rtt8OVr1Vb 0p2G5NaEJGUi7tcLR33WQBEpb9HJS7Cm2BXRjm7f0BqNHDDz6e3uS5OtWmzPPw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1682678424; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=FTss0UYWKma8Gaxrt5W8VgWbf1W7v05cjnJYCttJh08=; b=S/L412VOt9vVrMIWXO0yf4rvNtRGvIJG5420Kvi7Uznf6WIyMie6VdxstPHJXq6H/ATQsV xfndfcY3tDYMOSxLl8PR6M6BQtNbmBvmna10rubZhLsWQohUkxPM9kFpE028QpdjPsQj1Z osMwVTBeF8TVDCo3rzOogmcCbWyRy/ziUDtVrrgKakfX8qZNfOxsuHpUTTbOgG2lN9FNQ/ MKRwtABeQwPm7xWcrhXTfZsjeE8UDPTB4dRcPOjfO9zD88s54jkwfxvT2b0V+ike9LBuIl QHWCr0oGBZ5bAesruFRTvjD3vY0Mbu3tXj2LzDfq7uC4YAWFtgov8JGh78831g== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1682678424; a=rsa-sha256; cv=none; b=U0+F1R7AfzXARiEa4GKCZL86FMDR7aXFSDEb3hTCscAo0n90zstGehUYhK4NsIKPzj3tNR V/nvxO8VGklM6dz5u/KdWNbgOlfLDWS3mwNqDbSREgMwzrREf5gDJb7eiLfgBQipQ7OZAc ErMIflsocLziN1QlRig6TW6jEUjJN8xBpERyq8ZyQNHCTfJXkRvHooLi6JKQQHXWFhqIyC 8iC7hOfY020j0TlWBXLfbr0nvYOIs6Voaex/f+ms0/GCt8bbxOmlxlBGvjeaoTdtjZCphP B4mwo/GbCpKmkzYGVJlI/1JwSc4cVWGoBYibL3FJnAZvjZPaqMZ+FwIJqrYW8Q== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4Q78JS0Hl0zwLc; Fri, 28 Apr 2023 10:40:24 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 33SAeNj9092631; Fri, 28 Apr 2023 10:40:23 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 33SAeNuj092630; Fri, 28 Apr 2023 10:40:23 GMT (envelope-from git) Date: Fri, 28 Apr 2023 10:40:23 GMT Message-Id: <202304281040.33SAeNuj092630@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: =?utf-8?Q?Corvin=20K=C3=B6hne?= Subject: git: 0ad27bd40b35 - stable/13 - bhyve: add hook for PCI header of passthru devices List-Id: Commits to the stable branches of the FreeBSD src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-branches List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-branches@freebsd.org X-BeenThere: dev-commits-src-branches@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: corvink X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 0ad27bd40b351403a6fd0b58eafa98a43407cc83 Auto-Submitted: auto-generated X-ThisMailContainsUnwantedMimeParts: N The branch stable/13 has been updated by corvink: URL: https://cgit.FreeBSD.org/src/commit/?id=0ad27bd40b351403a6fd0b58eafa98a43407cc83 commit 0ad27bd40b351403a6fd0b58eafa98a43407cc83 Author: Corvin Köhne AuthorDate: 2021-09-07 11:42:25 +0000 Commit: Corvin Köhne CommitDate: 2023-04-28 07:28:34 +0000 bhyve: add hook for PCI header of passthru devices Most register of the PCI header are either constant values or require emulation anyway. The command and status register are the only exception which require hardware access. So, we're adding an emulation handler for all other register. As this emulation handler will be reused by some future features like GPU passthrough, we directly export it. Reviewed by: markj MFC after: 1 week Sponsored by: Beckhoff Automation GmbH & Co. KG Differential Revision: https://reviews.freebsd.org/D33010 (cherry picked from commit b6e67875a3e5bc706fe37565e464c61fe7833846) --- usr.sbin/bhyve/pci_passthru.c | 71 +++++++++++++++++++++++++------------------ usr.sbin/bhyve/pci_passthru.h | 4 +++ 2 files changed, 45 insertions(+), 30 deletions(-) diff --git a/usr.sbin/bhyve/pci_passthru.c b/usr.sbin/bhyve/pci_passthru.c index 79068ad9b6da..b214021998d2 100644 --- a/usr.sbin/bhyve/pci_passthru.c +++ b/usr.sbin/bhyve/pci_passthru.c @@ -603,6 +603,7 @@ cfginit(struct pci_devinst *pi, int bus, int slot, int func) { int error; struct passthru_softc *sc; + uint8_t intline, intpin; error = 1; sc = pi->pi_arg; @@ -612,6 +613,19 @@ cfginit(struct pci_devinst *pi, int bus, int slot, int func) sc->psc_sel.pc_dev = slot; sc->psc_sel.pc_func = func; + /* + * Copy physical PCI header to virtual config space. INTLINE and INTPIN + * shouldn't be aligned with their physical value and they are already set by + * pci_emul_init(). + */ + intline = pci_get_cfgdata8(pi, PCIR_INTLINE); + intpin = pci_get_cfgdata8(pi, PCIR_INTPIN); + for (int i = 0; i <= PCIR_MAXLAT; i += 4) { + pci_set_cfgdata32(pi, i, read_config(&sc->psc_sel, i, 4)); + } + pci_set_cfgdata8(pi, PCIR_INTLINE, intline); + pci_set_cfgdata8(pi, PCIR_INTPIN, intpin); + if (cfginitmsi(sc) != 0) { warnx("failed to initialize MSI for PCI %d/%d/%d", bus, slot, func); @@ -876,6 +890,15 @@ passthru_init(struct pci_devinst *pi, nvlist_t *nvl) get_config_value_node(nvl, "rom"))) != 0) goto done; + /* Emulate most PCI header register. */ + if ((error = set_pcir_handler(sc, 0, PCIR_MAXLAT + 1, + passthru_cfgread_emulate, passthru_cfgwrite_emulate)) != 0) + goto done; + + /* Allow access to the physical command and status register. */ + if ((error = set_pcir_handler(sc, PCIR_COMMAND, 0x04, NULL, NULL)) != 0) + goto done; + error = 0; /* success */ done: if (error) { @@ -885,16 +908,6 @@ done: return (error); } -static int -bar_access(int coff) -{ - if ((coff >= PCIR_BAR(0) && coff < PCIR_BAR(PCI_BARMAX + 1)) || - coff == PCIR_BIOS) - return (1); - else - return (0); -} - static int msicap_access(struct passthru_softc *sc, int coff) { @@ -926,23 +939,11 @@ passthru_cfgread_default(struct passthru_softc *sc, struct pci_devinst *pi __unused, int coff, int bytes, uint32_t *rv) { /* - * PCI BARs and MSI capability is emulated. + * MSI capability is emulated. */ - if (bar_access(coff) || msicap_access(sc, coff) || - msixcap_access(sc, coff)) + if (msicap_access(sc, coff) || msixcap_access(sc, coff)) return (-1); -#ifdef LEGACY_SUPPORT - /* - * Emulate PCIR_CAP_PTR if this device does not support MSI capability - * natively. - */ - if (sc->psc_msi.emulated) { - if (coff >= PCIR_CAP_PTR && coff < PCIR_CAP_PTR + 4) - return (-1); - } -#endif - /* * Emulate the command register. If a single read reads both the * command and status registers, read the status register from the @@ -962,6 +963,14 @@ passthru_cfgread_default(struct passthru_softc *sc, return (0); } +int +passthru_cfgread_emulate(struct passthru_softc *sc __unused, + struct pci_devinst *pi __unused, int coff __unused, int bytes __unused, + uint32_t *rv __unused) +{ + return (-1); +} + static int passthru_cfgread(struct pci_devinst *pi, int coff, int bytes, uint32_t *rv) { @@ -982,12 +991,6 @@ passthru_cfgwrite_default(struct passthru_softc *sc, struct pci_devinst *pi, int error, msix_table_entries, i; uint16_t cmd_old; - /* - * PCI BARs are emulated - */ - if (bar_access(coff)) - return (-1); - /* * MSI capability is emulated */ @@ -1054,6 +1057,14 @@ passthru_cfgwrite_default(struct passthru_softc *sc, struct pci_devinst *pi, return (0); } +int +passthru_cfgwrite_emulate(struct passthru_softc *sc __unused, + struct pci_devinst *pi __unused, int coff __unused, int bytes __unused, + uint32_t val __unused) +{ + return (-1); +} + static int passthru_cfgwrite(struct pci_devinst *pi, int coff, int bytes, uint32_t val) { diff --git a/usr.sbin/bhyve/pci_passthru.h b/usr.sbin/bhyve/pci_passthru.h index f6ce2d4b85ab..e999e84f85f2 100644 --- a/usr.sbin/bhyve/pci_passthru.h +++ b/usr.sbin/bhyve/pci_passthru.h @@ -20,5 +20,9 @@ typedef int (*cfgwrite_handler)(struct passthru_softc *sc, uint32_t read_config(const struct pcisel *sel, long reg, int width); void write_config(const struct pcisel *sel, long reg, int width, uint32_t data); +int passthru_cfgread_emulate(struct passthru_softc *sc, struct pci_devinst *pi, + int coff, int bytes, uint32_t *rv); +int passthru_cfgwrite_emulate(struct passthru_softc *sc, struct pci_devinst *pi, + int coff, int bytes, uint32_t val); int set_pcir_handler(struct passthru_softc *sc, int reg, int len, cfgread_handler rhandler, cfgwrite_handler whandler);