From nobody Tue Aug 05 14:03:16 2025 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4bxFWT1S6hz63GLl; Tue, 05 Aug 2025 14:03:17 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4bxFWS6Vryz3FQM; Tue, 05 Aug 2025 14:03:16 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1754402596; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Y6srb2kGt5XKiRFzQiocBhLevu4lVicsXbGoJBzH2Bc=; b=VBInd6hbjriE9rnaBxb7DPA+VwCLMAuxX6C0hXlqD5W7+H2b2qZaw/sCEPcQUCq5mUsvNW 27Pm2grmI8x2tP1tKfaOBox+iQWm7eC6xKYhNeMBZGirpaYyvj/ShELP7b/Y7vtZudS7J+ MBOufY/k7esyVMI7Qjyo3qbUsbhFK/g/5IwEt4Pv14TFkvNItDBNZHBbPTd+OTRd7yEgbf v5jylTfMMfavXG5wkHNF0dgtFnNlJesrsWGLH96+qWovxkHSOMLMgIsO7tgLo6Mx8g1Eqp DD6BP2JvFvOXVlgpJjjepKT6XygxQwBny9PyqI0gjVoN1CkmbewGEENdzi5pgQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1754402596; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Y6srb2kGt5XKiRFzQiocBhLevu4lVicsXbGoJBzH2Bc=; b=OArbaDGB6JH3O+oK+FiPC6jIGc/XKOFU7xNyxAoGCegoMXUUZ6HnKtJ1Fjt/gEU+DUeBW/ 8iBGQ0JBIs6KFNrlpO0MUHQ9s5q/DXBkXmNopyzbys3SzW+lZrkpifhPgtC5cxDlrgC5Ya HcvtA6H9gekm29/RENvyjk/rAfsZ7/bgXnMlUgqB7xOkoCoJZfryihJFvAAGROpc6zagnx ZxDyq6fy7EZRlGPdLNf+XpUk22/UulWKdL7QaXK/2dkTlNS+j8KIzbqZjPomAz+ZAnGx5q aeXykWmZymV3oAUbpxmFOYjthcDLZ40pgtIAhZ3srl1VkC5iE7kfqoMLdNLkdg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1754402596; a=rsa-sha256; cv=none; b=U9oJRcOW40xI3tXCfUHVDILPfVeraV3Ck7HNPYRBAhHPgxqn5JRuhq6k9vowDHzcp5YXrG P2uBJvh0WcnPEocyY19d2ksCxdmjcUgNngH8BlahrJIF8dYdm3rzdIX2flqjLKQdNYKNZA l6t+9n9dctPdya95LCNQeK69VUGNcO07lsLlmW2S9v+6LbC6GGnqPtXDXGdMOTcsu4S4hw ZdDteDYxtDMppqpvnbebrwoCzrA1dtet30ZyfTRhBpahWoTcuzr45J2ylcxL3yDP5ZIjkm e1ODe8Mmv2nl/qnim/y5AoQ9NaPKNHv9VEqLDKSFOJzXlCrzOw7irB03Z7IgJg== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4bxFWS65BGzfK3; Tue, 05 Aug 2025 14:03:16 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 575E3GIN070601; Tue, 5 Aug 2025 14:03:16 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 575E3GEj070598; Tue, 5 Aug 2025 14:03:16 GMT (envelope-from git) Date: Tue, 5 Aug 2025 14:03:16 GMT Message-Id: <202508051403.575E3GEj070598@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Corvin =?utf-8?Q?K=C3=B6hne?= Subject: git: df6c805ca125 - main - bhyve: add BAR handler list for passthru devices List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: corvink X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: df6c805ca12527d87051136ee5cf32f28e32d8cb Auto-Submitted: auto-generated The branch main has been updated by corvink: URL: https://cgit.FreeBSD.org/src/commit/?id=df6c805ca12527d87051136ee5cf32f28e32d8cb commit df6c805ca12527d87051136ee5cf32f28e32d8cb Author: Corvin Köhne AuthorDate: 2024-01-08 13:55:49 +0000 Commit: Corvin Köhne CommitDate: 2025-08-05 14:02:08 +0000 bhyve: add BAR handler list for passthru devices We have to emulate some specific register of a BAR for passthru devices. Our current use case are Intels integrated graphic devices. They mirror some of their PCI config space into the MMIO space. Unfortunately, the Windows driver reads from MMIO instead of PCI config space. For that reason, we have to trap and emulate those register. Instead of implementing a quirk for this special device, we're implementing a generic approach by using a list of trapped register. That's much cleaner and can be reused. E.g. Nvidia GPUs mirror their PCI config space in MMIO too and we can reuse it to trap the MSI-X table in the future. Note that the handling of this new list requires a larger patch. For that reason, we split it into multiple commits. This means that the list isn't used yet. This commit adds the callback on BAR reads and writes. Some subsequent commit will add the trap for BAR regions and an interface to easily add protected regions. Reviewed by: jhb, markj MFC after: 1 week Sponsored by: Beckhoff Automation GmbH & Co. KG Differential Revision: https://reviews.freebsd.org/D45339 --- usr.sbin/bhyve/pci_passthru.c | 59 ++++++++++++++++++++++++++++++++++++++++--- usr.sbin/bhyve/pci_passthru.h | 4 +++ 2 files changed, 59 insertions(+), 4 deletions(-) diff --git a/usr.sbin/bhyve/pci_passthru.c b/usr.sbin/bhyve/pci_passthru.c index a82078f6e036..0cc5b777b842 100644 --- a/usr.sbin/bhyve/pci_passthru.c +++ b/usr.sbin/bhyve/pci_passthru.c @@ -79,6 +79,14 @@ static int pcifd = -1; SET_DECLARE(passthru_dev_set, struct passthru_dev); +struct passthru_bar_handler { + TAILQ_ENTRY(passthru_bar_handler) chain; + uint64_t off; + uint64_t size; + passthru_read_handler read; + passthru_write_handler write; +}; + struct passthru_softc { struct pci_devinst *psc_pi; /* ROM is handled like a BAR */ @@ -96,6 +104,9 @@ struct passthru_softc { struct passthru_mmio_mapping psc_mmio_map[PASSTHRU_MMIO_MAX]; cfgread_handler psc_pcir_rhandler[PCI_REGMAX + 1]; cfgwrite_handler psc_pcir_whandler[PCI_REGMAX + 1]; + + TAILQ_HEAD(, + passthru_bar_handler) psc_bar_handler[PCI_BARMAX_WITH_ROM + 1]; }; static int @@ -947,6 +958,9 @@ passthru_init(struct pci_devinst *pi, nvlist_t *nvl) pi->pi_arg = sc; sc->psc_pi = pi; + for (uint8_t i = 0; i < PCI_BARMAX_WITH_ROM + 1; ++i) + TAILQ_INIT(&sc->psc_bar_handler[i]); + /* initialize config space */ if ((error = cfginit(pi, bus, slot, func)) != 0) goto done; @@ -1166,6 +1180,7 @@ passthru_write(struct pci_devinst *pi, int baridx, uint64_t offset, int size, uint64_t value) { struct passthru_softc *sc; + struct passthru_bar_handler *handler; struct pci_bar_ioreq pio; sc = pi->pi_arg; @@ -1173,9 +1188,27 @@ passthru_write(struct pci_devinst *pi, int baridx, uint64_t offset, int size, if (baridx == pci_msix_table_bar(pi)) { msix_table_write(sc, offset, size, value); } else { - assert(pi->pi_bar[baridx].type == PCIBAR_IO); assert(size == 1 || size == 2 || size == 4); - assert(offset <= UINT32_MAX && offset + size <= UINT32_MAX); + + TAILQ_FOREACH(handler, &sc->psc_bar_handler[baridx], chain) { + if (offset >= handler->off + handler->size) { + continue; + } else if (offset < handler->off) { + assert(offset + size < handler->off); + /* + * The list is sorted in ascending order, so all + * remaining handlers will have an even larger + * offset. + */ + break; + } + + assert(offset + size <= handler->off + handler->size); + + handler->write(pi, baridx, + offset - handler->off, size, value); + return; + } bzero(&pio, sizeof(pio)); pio.pbi_sel = sc->psc_sel; @@ -1193,6 +1226,7 @@ static uint64_t passthru_read(struct pci_devinst *pi, int baridx, uint64_t offset, int size) { struct passthru_softc *sc; + struct passthru_bar_handler *handler; struct pci_bar_ioreq pio; uint64_t val; @@ -1201,9 +1235,26 @@ passthru_read(struct pci_devinst *pi, int baridx, uint64_t offset, int size) if (baridx == pci_msix_table_bar(pi)) { val = msix_table_read(sc, offset, size); } else { - assert(pi->pi_bar[baridx].type == PCIBAR_IO); assert(size == 1 || size == 2 || size == 4); - assert(offset <= UINT32_MAX && offset + size <= UINT32_MAX); + + TAILQ_FOREACH(handler, &sc->psc_bar_handler[baridx], chain) { + if (offset >= handler->off + handler->size) { + continue; + } else if (offset < handler->off) { + assert(offset + size < handler->off); + /* + * The list is sorted in ascending order, so all + * remaining handlers will have an even larger + * offset. + */ + break; + } + + assert(offset + size <= handler->off + handler->size); + + return (handler->read(pi, baridx, + offset - handler->off, size)); + } bzero(&pio, sizeof(pio)); pio.pbi_sel = sc->psc_sel; diff --git a/usr.sbin/bhyve/pci_passthru.h b/usr.sbin/bhyve/pci_passthru.h index a89ad287cbc5..fc3fdbc1719f 100644 --- a/usr.sbin/bhyve/pci_passthru.h +++ b/usr.sbin/bhyve/pci_passthru.h @@ -35,6 +35,10 @@ typedef int (*cfgread_handler)(struct passthru_softc *sc, struct pci_devinst *pi, int coff, int bytes, uint32_t *rv); typedef int (*cfgwrite_handler)(struct passthru_softc *sc, struct pci_devinst *pi, int coff, int bytes, uint32_t val); +typedef uint64_t (*passthru_read_handler)(struct pci_devinst *pi, int baridx, + uint64_t offset, int size); +typedef void (*passthru_write_handler)(struct pci_devinst *pi, int baridx, uint64_t offset, + int size, uint64_t val); uint32_t pci_host_read_config(const struct pcisel *sel, long reg, int width); void pci_host_write_config(const struct pcisel *sel, long reg, int width,