[Bug 299136] bhyve(8): GVT-d code claims discrete Intel Arc GPUs, so PCI passthrough using them fails

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=299136

            Bug ID: 299136
           Summary: bhyve(8): GVT-d code claims discrete Intel Arc GPUs,
                    so PCI passthrough using them fails
           Product: Base System
           Version: 15.1-RELEASE
          Hardware: amd64
                OS: Any
            Status: New
          Severity: Affects Some People
          Priority: ---
         Component: bhyve
          Assignee: virtualization@FreeBSD.org
          Reporter: eppower@umich.edu

Created attachment 275388
  --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=275388&action=edit
This patch prevents gvt_d_probe from erroneously treating Intel dGPUs in the
same way it handles iGPUs and instead dGPUs fall back to simple passthrough.

Disclaimer: AI (Claude Opus 5.5) was used to help me solve a dGPU passthru
issue using an Intel Arc A310 and bhyve - I believe it should help for any
Intel dGPU passthrough scenario. This is related to bug 299135, in that the
symptoms were simultaneously observed. I am not a software developer and could
not have developed this fix on my own. For my setup, this fix works and allows
me to start a VM with a passthrough Intel dGPU. I've reviewed the
Claude-generated bug report below and believe it to be factual and accurate,
however if you find that you need additional information I'd of course be happy
to supply anything I can.

Description:

I believe this to be similar to bug 296663.

usr.sbin/bhyve/amd64/pci_gvt-d.c attaches its GVT-d handling to any
passthrough device whose vendor is Intel and whose class is PCIC_DISPLAY
(gvt_d_probe()). Discrete Intel GPUs (Arc / DG2) match too. bhyve then
runs the integrated-GPU setup on them (graphics stolen memory, OpRegion,
BDSM), which fails, and the VM does not start.

On a host with no Intel integrated graphics, the stolen-memory sysctls
report zero and passthrough of an Arc A310 (8086:56a6, subsystem
172f:4019) fails with:

    bhyve: gvt_d_setup_gsm: Unable to add Graphics Stolen Memory to E820 table
(hpa 0x0 len 0x0)
    E820 map:
      (   0) [               0,                0] Reserved
      (   1) [               0,            a0000] RAM
      (   2) [          100000,         10000000] RAM
    bhyve: gvt_d_init: Unable to setup Graphics Stolen Memory
    Device emulation initialization error: No such file or directory

and bhyve exits with status 4. Passing through only the GPU's HDA function
works, since it is not a display-class device.

The same failure mode was reported on freebsd-virtualization in January
2025 for an Arc A380 on an AMD EPYC host (OpRegion errors in that case):
https://lists.freebsd.org/archives/freebsd-virtualization/2025-January/002659.html

The probe is identical in releng/14.4, releng/15.1 and main (checked early
October 2026). The DG2 device IDs are already present in
usr.sbin/bhyve/pciids_intel_gpus.h, but they are not in the igd_devices[]
table that the GVT-d code supports (which ends at Raptor Lake), so
get_igd_ops() returns NULL for them.

Environment:

- FreeBSD 15.1-RELEASE amd64
- 2x Intel Xeon Platinum 8260 (no integrated graphics), Supermicro
  X11DPH-i, VT-d enabled
- Sparkle Intel Arc A310 ECO 4GB low-profile (8086:56a6, 172f:4019),
  GPU at pci0:136:0:0, HDA at pci0:137:0:0, both reserved via pptdevs

How to reproduce:

    bhyve -c 1 -m 256M -AHPw -S \
      -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
      -s 0,hostbridge -s 31,lpc \
      -s 2:0,passthru,136/0/0 -s 3:0,passthru,137/0/0 \
      testvm

bhyve exits within a second or two with the errors above.

Fix:

The attached patch (gvtd-skip-discrete.patch, against releng/15.1) makes
gvt_d_probe() return ENXIO when get_igd_ops() finds no entry for the
device, so only integrated GPUs listed in igd_devices[] are claimed by
GVT-d. Other Intel display devices fall back to plain passthrough.

Testing with the patch:

- The same command keeps running; the GPU and its HDA function are
  assigned to the guest.
- An Ubuntu 24.04 live guest with the GPU passed through: i915 binds and
  fully initializes the card (DMC, GuC and HuC firmware load, "GUC:
  submission enabled", "HuC: authenticated for all workloads", HDA
  component bound, MEI/GSC interface bound).
- From that Linux guest, Intel's igsc tool read and updated the card's GSC
  firmware (DG02_2.2357 -> DG02_2.2359) through passthrough.

Attachment: gvtd-skip-discrete.patch

Back Matter: Here's a bit of text copied from bug report 299135 to provide you
with the human context- I have a Supermicro X11DPH-I mainboard (dual Cascade
Lake Xeon board) with a discrete Intel X710-T2L 10GbE NIC installed. The X710
supports SR-IOV, and I use multiple VFs on this NIC to support my jail
networking. I also have a Sparkle A310 ECO GPU installed, which is an Intel
Alchemist generation dGPU. The GPU is intended to provide hardware-accelerated
transcoding for a VM running jellyfin. The host machine was originally brought
online with 14.3-RELEASE, then upgraded to 14.4-RELEASE, and most recently
upgraded to 15.1-RELEASE. Across all versions I noted the same behavior - when
attempting to pass through the GPU to a VM (either with a native bhyve call or
using vm-bhyve), all VFs on my NIC immediately died a horrible death with
messages like "Malicious driver detection" and complete loss of networking
functionality. Bug 299135 addresses the loss of all VFs when bhyve starts up -
this bug report relates to the observation that bhyve failed to start a VM
successfully with the GPU attached - the VM would crash in < 4 seconds, leaving
me without a working VM (and a set of jails orphaned from their network
adapters).

-- 
You are receiving this mail because:
You are the assignee for the bug.