From nobody Wed Apr 12 07:22:37 2023 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PxDgf3HJDz45R8W; Wed, 12 Apr 2023 07:22:38 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PxDgd6C8nz4Hnp; Wed, 12 Apr 2023 07:22:37 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1681284157; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=QQHGJVyyeE/V5zYcPAR9X8zgNgNaNvxbtME+df3w/ig=; b=nB6tH0HLq4UvEaUa8rIFWUA67cdN4xXb3APTZOlZQqVq5AyGiDtvnf5bA5tVLTL9vudROr y9vjepUUb/jFarKr9yvwBiD4Br7rfMRDTI1uHCu0fk6R7LU2UBl97sNj2Juidv7J9mHRkz XBz7MDVcD17kBOo5c4BVxR1sgE8RUs/yJX+hX1FEy3JqGHm5AN6/O6WUH56BwlsT8SlvIY A58j8400OnMZMShsCu8j3QD2LQwzTngwN97rLrA4ocM2nzrowsW5LXnEqV3Yq71K9ik2HR IlgalQOQrq/HjtkOrQPDDflKqGdpfGLEWn1Y/c4lH6uXKOLZTlaRVwVBUJgXng== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1681284157; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=QQHGJVyyeE/V5zYcPAR9X8zgNgNaNvxbtME+df3w/ig=; b=LL73Xii61O9+Lrv71BBvsJEfCxhnELbaSmZVe8cM25kbanVcxX2nMMWFnk+TatRbS/vCNs 68+bU10bgRy7DzWSMeWgMXm6nyMalKPolyxQblK8NbujcRc+yd4ttvQhW4uF1zurr9xfJ7 fqdsIqlW2KwLOP4ED5YIpTXhzor4Uzfb2yJkNb78BNhJSmIxK5E8ULdVHbCd3a0lB36ZYx pvdEb3d4qVpuYfWz54RkT5h7WR51iq0JLt8HjTIjOuVkBsiw1Ki2shRJWQq9ME9bnOyCgU y17wyNkqUkWasEpN0gitMILcc2IlBA18sY6SNZn/kJ3Gy3mWHN81bLQ5ieVUVw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1681284157; a=rsa-sha256; cv=none; b=jW/UWGsXB7uqpUgxz+6PTNfS5s7Z0BKaXE07SZATcDDWtqU76Y4gfJbSd6m7PuKP1Ccl3u tyhYrppOvvST8+a+MZkX7oIdac6IERpsS0SZSg9wLpeTneJXpNtdBzwm+k3xhLfwym04FQ 6QUKeGRw5a3RtW014tzLDV1QlvlV+72QpvzAipLbtOJHbL38idwkhKWLll8zfWJFon2Hqp e8foaHlU0rywEzv0VK3XQxLlCXfB8Qdp5xshhbIlsn8Ym8dlQrqP7umpz9ZVlU3yIU61PE eo2K3LfBBd48X30eFSekkmAKPyizQsIeRuvo8WPrwmbh2BX3SzreVr4uGH0Xwg== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4PxDgd3krhzrDX; Wed, 12 Apr 2023 07:22:37 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 33C7MbW9094677; Wed, 12 Apr 2023 07:22:37 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 33C7MbGd094676; Wed, 12 Apr 2023 07:22:37 GMT (envelope-from git) Date: Wed, 12 Apr 2023 07:22:37 GMT Message-Id: <202304120722.33C7MbGd094676@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: =?utf-8?Q?Corvin=20K=C3=B6hne?= Subject: git: b6e67875a3e5 - main - bhyve: add hook for PCI header of passthru devices List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-main@freebsd.org X-BeenThere: dev-commits-src-main@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: corvink X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: b6e67875a3e5bc706fe37565e464c61fe7833846 Auto-Submitted: auto-generated X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by corvink: URL: https://cgit.FreeBSD.org/src/commit/?id=b6e67875a3e5bc706fe37565e464c61fe7833846 commit b6e67875a3e5bc706fe37565e464c61fe7833846 Author: Corvin Köhne AuthorDate: 2021-09-07 11:42:25 +0000 Commit: Corvin Köhne CommitDate: 2023-04-12 07:22:24 +0000 bhyve: add hook for PCI header of passthru devices Most register of the PCI header are either constant values or require emulation anyway. The command and status register are the only exception which require hardware access. So, we're adding an emulation handler for all other register. As this emulation handler will be reused by some future features like GPU passthrough, we directly export it. Reviewed by: markj MFC after: 1 week Sponsored by: Beckhoff Automation GmbH & Co. KG Differential Revision: https://reviews.freebsd.org/D33010 --- usr.sbin/bhyve/pci_passthru.c | 71 +++++++++++++++++++++++++------------------ usr.sbin/bhyve/pci_passthru.h | 4 +++ 2 files changed, 45 insertions(+), 30 deletions(-) diff --git a/usr.sbin/bhyve/pci_passthru.c b/usr.sbin/bhyve/pci_passthru.c index 0055de461270..b5c83f302f0e 100644 --- a/usr.sbin/bhyve/pci_passthru.c +++ b/usr.sbin/bhyve/pci_passthru.c @@ -603,6 +603,7 @@ cfginit(struct pci_devinst *pi, int bus, int slot, int func) { int error; struct passthru_softc *sc; + uint8_t intline, intpin; error = 1; sc = pi->pi_arg; @@ -612,6 +613,19 @@ cfginit(struct pci_devinst *pi, int bus, int slot, int func) sc->psc_sel.pc_dev = slot; sc->psc_sel.pc_func = func; + /* + * Copy physical PCI header to virtual config space. INTLINE and INTPIN + * shouldn't be aligned with their physical value and they are already set by + * pci_emul_init(). + */ + intline = pci_get_cfgdata8(pi, PCIR_INTLINE); + intpin = pci_get_cfgdata8(pi, PCIR_INTPIN); + for (int i = 0; i <= PCIR_MAXLAT; i += 4) { + pci_set_cfgdata32(pi, i, read_config(&sc->psc_sel, i, 4)); + } + pci_set_cfgdata8(pi, PCIR_INTLINE, intline); + pci_set_cfgdata8(pi, PCIR_INTPIN, intpin); + if (cfginitmsi(sc) != 0) { warnx("failed to initialize MSI for PCI %d/%d/%d", bus, slot, func); @@ -876,6 +890,15 @@ passthru_init(struct pci_devinst *pi, nvlist_t *nvl) get_config_value_node(nvl, "rom"))) != 0) goto done; + /* Emulate most PCI header register. */ + if ((error = set_pcir_handler(sc, 0, PCIR_MAXLAT + 1, + passthru_cfgread_emulate, passthru_cfgwrite_emulate)) != 0) + goto done; + + /* Allow access to the physical command and status register. */ + if ((error = set_pcir_handler(sc, PCIR_COMMAND, 0x04, NULL, NULL)) != 0) + goto done; + error = 0; /* success */ done: if (error) { @@ -885,16 +908,6 @@ done: return (error); } -static int -bar_access(int coff) -{ - if ((coff >= PCIR_BAR(0) && coff < PCIR_BAR(PCI_BARMAX + 1)) || - coff == PCIR_BIOS) - return (1); - else - return (0); -} - static int msicap_access(struct passthru_softc *sc, int coff) { @@ -926,23 +939,11 @@ passthru_cfgread_default(struct passthru_softc *sc, struct pci_devinst *pi __unused, int coff, int bytes, uint32_t *rv) { /* - * PCI BARs and MSI capability is emulated. + * MSI capability is emulated. */ - if (bar_access(coff) || msicap_access(sc, coff) || - msixcap_access(sc, coff)) + if (msicap_access(sc, coff) || msixcap_access(sc, coff)) return (-1); -#ifdef LEGACY_SUPPORT - /* - * Emulate PCIR_CAP_PTR if this device does not support MSI capability - * natively. - */ - if (sc->psc_msi.emulated) { - if (coff >= PCIR_CAP_PTR && coff < PCIR_CAP_PTR + 4) - return (-1); - } -#endif - /* * Emulate the command register. If a single read reads both the * command and status registers, read the status register from the @@ -962,6 +963,14 @@ passthru_cfgread_default(struct passthru_softc *sc, return (0); } +int +passthru_cfgread_emulate(struct passthru_softc *sc __unused, + struct pci_devinst *pi __unused, int coff __unused, int bytes __unused, + uint32_t *rv __unused) +{ + return (-1); +} + static int passthru_cfgread(struct pci_devinst *pi, int coff, int bytes, uint32_t *rv) { @@ -982,12 +991,6 @@ passthru_cfgwrite_default(struct passthru_softc *sc, struct pci_devinst *pi, int error, msix_table_entries, i; uint16_t cmd_old; - /* - * PCI BARs are emulated - */ - if (bar_access(coff)) - return (-1); - /* * MSI capability is emulated */ @@ -1054,6 +1057,14 @@ passthru_cfgwrite_default(struct passthru_softc *sc, struct pci_devinst *pi, return (0); } +int +passthru_cfgwrite_emulate(struct passthru_softc *sc __unused, + struct pci_devinst *pi __unused, int coff __unused, int bytes __unused, + uint32_t val __unused) +{ + return (-1); +} + static int passthru_cfgwrite(struct pci_devinst *pi, int coff, int bytes, uint32_t val) { diff --git a/usr.sbin/bhyve/pci_passthru.h b/usr.sbin/bhyve/pci_passthru.h index f6ce2d4b85ab..e999e84f85f2 100644 --- a/usr.sbin/bhyve/pci_passthru.h +++ b/usr.sbin/bhyve/pci_passthru.h @@ -20,5 +20,9 @@ typedef int (*cfgwrite_handler)(struct passthru_softc *sc, uint32_t read_config(const struct pcisel *sel, long reg, int width); void write_config(const struct pcisel *sel, long reg, int width, uint32_t data); +int passthru_cfgread_emulate(struct passthru_softc *sc, struct pci_devinst *pi, + int coff, int bytes, uint32_t *rv); +int passthru_cfgwrite_emulate(struct passthru_softc *sc, struct pci_devinst *pi, + int coff, int bytes, uint32_t val); int set_pcir_handler(struct passthru_softc *sc, int reg, int len, cfgread_handler rhandler, cfgwrite_handler whandler);