From nobody Wed Nov 04 17:34:38 2026 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4jKXct611zz6yZ8m for ; Wed, 04 Nov 2026 17:34:38 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4jKXct5Mznz501Y for ; Wed, 04 Nov 2026 17:34:38 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1793813678; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=MhV0IrJdQjDZ47+A8fhTweQif7A4FXObkRr27duQG3k=; b=Xqkb+9yGCS5RaWqmr+XVzShfmug6XSFGEOQSWo5h4uIvm8A7kyGXuZ2SNSKlVUz3suo3ss 0sefTkEmal53xXGaygiiX0wl8YdeSOiguHsDnvVSAVqdJnGThFViNfycuDmM9OJgLldqB2 vmM/QFhH9QPsjY2eENcQYYRCklLzVUQ6vFTIf2A0McWfaIQN1sVOCGWrFk3pJJFx7BiVUB evCm/ZQFJVMpPEr9bP1zeKlATpAPm2GIajif/ImFiFdgmPCnhAvzpQHkmS9Fha6nG3GEt4 4DifjnkHx2JMvv1cZm067fKtjAJfh5Jv9GqlQAdMJlsegBVnNBXmJP6/8TMKPw== ARC-Seal: i=1; a=rsa-sha256; d=freebsd.org; s=dkim; cv=none; t=1793813678; b=n7tOLmHRUuAk2UZS3HJS89vSWiImr/gPIXG+yK1t2Do4PNNVLc9uK39TzoSkW328Z1R1gw HE4E3EaSF5DQ9nSa/k2IUsHMbxfnSQZQ3E/V5eyTSKW8jkvuj/cNpUbYMCsg1pxZ3BHvP6 mLcBFq86bfE4aYC7RstuTEB/AmyJ28hJ1fMDeb9j+1qKLL1U8kifKNaZzLkO3DTGKRgrJq Hh3U0et46BxM4uy55X0Fj15PbbgjvqTZG7aUSRjm0hE+LdU1230ra9co1+Qm6wtDX87LfJ zdYFX1/50pO7W0ACfrh41P201CQJcPfIMg3VO6JbKsSjOS5bOmuuxXqOGjRktg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1793813678; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=MhV0IrJdQjDZ47+A8fhTweQif7A4FXObkRr27duQG3k=; b=e3cZxOsd+3Z6kYrfAFKZQEJz1nPqP9OxRoS840sGFelgdrtTSXTeCJrl3I8/triqvuo2Nk RfqsQk4NATfSyYcwejd30VoxeR4SmekSA9C0bNO7d4NzXy3w3EdsnltAZw+gNNoUyDaAJI 58duFZ70uXiIXTQTtYN7A3LeWdhypTMNmbf0oIBERO3/A6N2Ms24MiltR92hIzKyLNMBFP +u2AgAC9Yrkm9RsGrpWQvhzpXuFDenRkj1i27agebpw8BoYgQCFv30DSREOI8q2MbGGY1x 7VdChPO65NpYdfVEf81LkrXelaX1vt0Q+FS7qaHvKWmX166di9c54B7gPzmfcw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4jKXct520Qz1CJ2 for ; Wed, 04 Nov 2026 17:34:38 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 26441 by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Wed, 04 Nov 2026 17:34:38 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org Cc: Wanpeng Qian From: Bojan Novk=?utf-8?Q?ovi=C4=87?= Subject: git: 75b1f9677268 - main - bhyve: restore the BARs at the addresses the guest gave them List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org List-Id: Commit messages for all branches of the src repository List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: bnovkov X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 75b1f96772688c4feb698bd5aece8793324b9e23 Auto-Submitted: auto-generated Date: Wed, 04 Nov 2026 17:34:38 +0000 Message-Id: <6aeb6cae.26441.18799713@gitrepo.freebsd.org> The branch main has been updated by bnovkov: URL: https://cgit.FreeBSD.org/src/commit/?id=75b1f96772688c4feb698bd5aece8793324b9e23 commit 75b1f96772688c4feb698bd5aece8793324b9e23 Author: Wanpeng Qian AuthorDate: 2026-11-04 17:32:43 +0000 Commit: Bojan Novković CommitDate: 2026-11-04 17:33:44 +0000 bhyve: restore the BARs at the addresses the guest gave them A restore set the addresses in the BARs and the command register to those of the snapshot, but left the regions registered where pci_emul_alloc_bar() had put them when bhyve started. That is only right for a guest that kept bhyve's addresses. The UEFI firmware moves 64-bit BARs above 4 GB, and after a restore every access to such a BAR went nowhere: Emulating access to non-existent address to 0x800001014 Before the devices are restored, unregister the regions of all of them; when a device is restored, register its regions at the restored addresses, if the restored command register enables them. All devices go first because a restored BAR may be where another device's BAR is before the restore. Signed-off-by: Wanpeng Qian Sponsored by: keelos.dev Reviewed by: bnovkov Differential Revision: https://reviews.freebsd.org/D60235 MFC after: 2 weeks Tested by: bnovkov --- usr.sbin/bhyve/pci_emul.c | 44 ++++++++++++++++++++++++++++++++++++++++++++ usr.sbin/bhyve/pci_emul.h | 1 + usr.sbin/bhyve/snapshot.c | 2 ++ 3 files changed, 47 insertions(+) diff --git a/usr.sbin/bhyve/pci_emul.c b/usr.sbin/bhyve/pci_emul.c index 60128a735de1..1ce9478bdebc 100644 --- a/usr.sbin/bhyve/pci_emul.c +++ b/usr.sbin/bhyve/pci_emul.c @@ -2510,6 +2510,48 @@ INOUT_PORT(pci_cfgdata, CONF1_DATA_PORT+3, IOPORT_F_INOUT, pci_emul_cfgdata); #endif #ifdef BHYVE_SNAPSHOT +/* + * Register or unregister the regions of the BARs that the device decodes. + */ +static void +pci_restore_register_bars(struct pci_devinst *pi, int registration) +{ + int decode, i; + + for (i = 0; i <= PCI_BARMAX; i++) { + switch (pi->pi_bar[i].type) { + case PCIBAR_IO: + decode = porten(pi); + break; + case PCIBAR_MEM32: + case PCIBAR_MEM64: + decode = memen(pi); + break; + default: + continue; + } + if (decode && pi->pi_bar[i].size != 0) + modify_bar_registration(pi, i, registration); + } +} + +/* + * To be called before the devices are restored. The regions of the BARs are + * registered at the addresses pci_emul_alloc_bar() chose; the restore of a + * device registers them at the addresses the guest had given them. All + * devices give up their regions first, because a restored BAR may be where + * the BAR of another device is now. + */ +void +pci_restore_prepare(void) +{ + struct pci_devinst *pi; + + pi = NULL; + while ((pi = pci_next(pi)) != NULL) + pci_restore_register_bars(pi, 0); +} + /* * Saves/restores PCI device emulated state. Returns 0 on success. */ @@ -2544,6 +2586,8 @@ pci_snapshot_pci_dev(struct vm_snapshot_meta *meta) SNAPSHOT_VAR_OR_LEAVE(pi->pi_bar[i].size, meta, ret, done); SNAPSHOT_VAR_OR_LEAVE(pi->pi_bar[i].addr, meta, ret, done); } + if (meta->op == VM_SNAPSHOT_RESTORE) + pci_restore_register_bars(pi, 1); /* Restore MSI-X table. */ for (i = 0; i < pi->pi_msix.table_count; i++) { diff --git a/usr.sbin/bhyve/pci_emul.h b/usr.sbin/bhyve/pci_emul.h index b7ddfd73bda1..d536e53d01e3 100644 --- a/usr.sbin/bhyve/pci_emul.h +++ b/usr.sbin/bhyve/pci_emul.h @@ -268,6 +268,7 @@ int pci_bus_configured(int bus); #ifdef BHYVE_SNAPSHOT struct pci_devinst *pci_next(const struct pci_devinst *cursor); int pci_snapshot(struct vm_snapshot_meta *meta); +void pci_restore_prepare(void); int pci_pause(struct pci_devinst *pdi); int pci_resume(struct pci_devinst *pdi); #endif diff --git a/usr.sbin/bhyve/snapshot.c b/usr.sbin/bhyve/snapshot.c index 1d3d337ec9b7..067b05622e68 100644 --- a/usr.sbin/bhyve/snapshot.c +++ b/usr.sbin/bhyve/snapshot.c @@ -876,6 +876,8 @@ vm_restore_devices(struct restore_state *rstate) int ret; struct pci_devinst *pdi = NULL; + pci_restore_prepare(); + while ((pdi = pci_next(pdi)) != NULL) { ret = vm_restore_device(rstate, pci_snapshot, pdi->pi_name, pdi); if (ret)