[Bug 300031] bhyve iGPU passthrough on Alder Lake-N: display engine faults on 10-bit/tiled GUI planes, i915 never gets vblank interrupts

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=300031

            Bug ID: 300031
           Summary: bhyve iGPU passthrough on Alder Lake-N: display engine
                    faults on 10-bit/tiled GUI planes, i915 never gets
                    vblank interrupts
           Product: Base System
           Version: 15.1-STABLE
          Hardware: amd64
                OS: Any
            Status: New
          Severity: Affects Some People
          Priority: ---
         Component: bhyve
          Assignee: virtualization@FreeBSD.org
          Reporter: michael.gruner@gmail.com

1. SUMMARY
----------

Passing the Intel Alder Lake-N UHD Graphics controller (PCI 8086:46d1) to a
bhyve Linux guest works for compute and for the initial display mode set: i915
binds, the DMC/GuC/HuC firmware loads, the HDMI connector reports "connected",
the EDID is read and a 1920x1080@60 mode is programmed. The boot/splash
surfaces are scanned out.

As soon as the guest's KMS client switches the GUI to a 10-bit XR30 overlay
plane with a tiled/compressed modifier, the display engine faults on the
surface fetch and no page flip ever completes. The i915 MSI interrupt counter
does not advance, i.e. the display engine raises no vblank interrupts. Result:
the splash renders, then the screen goes black.

2. ENVIRONMENT
--------------

Host:

    FreeBSD freebsd-ugreen 15.1-RELEASE-p4 GENERIC amd64
    Intel(R) N100  (Alder Lake-N), 16 GB RAM
    hw.vmm.iommu.enable: 1
    /boot/loader.conf:   pptdevs="0/2/0 0/20/0 0/31/3"

Relevant guest device (pciconf -lv on host):

    ppt0@pci0:0:2:0: class=0x030000 vendor=0x8086 device=0x46d1
        device = 'Alder Lake-N [UHD Graphics]'
    ppt2@pci0:0:31:3: class=0x040100 vendor=0x8086 device=0x54c8
        device = 'Alder Lake-N PCH High Definition Audio Controller'

vm-bhyve VM configuration (/zroot/vm/libreelec/libreelec.conf):

    loader="uefi"
    cpu="2"
    memory="5G"
    disk0_type="ahci-hd"
    disk0_dev="sparse-zvol"
    network0_type="virtio-net"
    network0_switch="public"
    passthru0="0/31/3=31:3"
    bhyve_options="-s
2:0,passthru,0/2/0,rom=/zroot/vm/libreelec/ADL-N_TWL_GOPv21_igd.rom -o
lpc.fwcfg=qemu"

Resulting bhyve command line (wrapped here for readability; it is a single
command line):

    -c 2 -m 5G -AHPw -l
bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd,.../uefi-vars.fd \
      -s 2:0,passthru,0/2/0,rom=.../ADL-N_TWL_GOPv21_igd.rom -o lpc.fwcfg=qemu
-S ... -u -S
    -s 0,hostbridge -s 31,lpc -s 0:4:0,ahci-hd,<disk> -s 0:5:0,virtio-net,<tap>
-s 0:31:3,passthru,0/31/3

Guest: LibreELEC 12.2.1, Linux 6.16.12, Kodi 21 (GBM/atomic), Mesa,
libva 2.22 / iHD 25.2.6.

Host firmware packages: gpu-firmware-intel-kmod-alderlake,
gpu-firmware-intel-kmod-tigerlake, edk2-bhyve-g202508_3.

3. HOW TO REPRODUCE
-------------------

1) Reserve the iGPU with pptdevs="0/2/0" and reboot the host.
2) Start the guest with the passthru configuration shown above (rom= GOP ROM
   and -o lpc.fwcfg=qemu).
3) Attach an HDMI display and boot the guest.
4) i915 binds and the connector reports connected; the boot splash / Kodi
   splash is visible on the display.
5) Kodi brings up its GUI and the display goes black.
6) Minimal reproducer: pass an ADL-N iGPU to a Linux guest, attach an HDMI
   display, start any KMS client that uses a 10-bit tiled plane (e.g. Kodi
   21), then watch /proc/interrupts for i915 and dmesg for plane faults.

4. OBSERVED GUEST STATE AND LOGS
--------------------------------

i915 bind and firmware load:

    i915 0000:00:02.0: [drm] Found alderlake_p/alderlake_n (device ID 46d1)
integrated display version 13.00 stepping D0
    i915 0000:00:02.0: [drm] Finished loading DMC firmware i915/adlp_dmc.bin
(v2.20)
    i915 0000:00:02.0: [drm] GT0: GuC firmware i915/tgl_guc_70.bin version
70.44.1
    i915 0000:00:02.0: [drm] GT0: HuC firmware i915/tgl_huc.bin version 7.9.3
    i915 0000:00:02.0: [drm] Initialized i915 1.6.0 for 0000:00:02.0 on minor 0

Connector and mode:

    /sys/class/drm/card0-HDMI-A-1/status: connected
    /sys/class/drm/card0-HDMI-A-1/edid:   256 bytes
    modes: 1920x1080 ...

i915_display_info (/sys/kernel/debug/dri/0/i915_display_info):

    [CRTC:88:pipe A]:
        uapi: enable=yes, active=yes, mode="1920x1080": 60 148500 ...
        hw: enable=yes, active=yes
        [ENCODER:204:DDI B/PHY B]: connectors:
            [CONNECTOR:205:HDMI-A-1]
        [PLANE:43:plane 2A]: type=OVL
            uapi: [FB:216] XR30 little-endian
(0x30335258),0x100000000000008,1920x1080, visible=visible, ...
            hw:   [FB:216] XR30 little-endian
(0x30335258),0x100000000000008,1920x1080, visible=yes, ...

Kernel log at the failure:

    i915 0000:00:02.0: [drm] *ERROR* [CRTC:88:pipe A][PLANE:43:plane 2A] fault
(CTL=0x92009000, SURF=0xfedb7000, SURFLIVE=0xfedb7000)
    i915 0000:00:02.0: [drm] *ERROR* flip_done timed out
    i915 0000:00:02.0: [drm] *ERROR* [PLANE:43:plane 2A] commit wait timed out
    i915 0000:00:02.0: [drm] *ERROR* [CRTC:88:pipe A] commit wait timed out
    i915 0000:00:02.0: [drm] *ERROR* [CONNECTOR:205:HDMI-A-1] commit wait timed
out

Interrupt delivery is frozen while flips are pending and an active display is
configured:

    # grep i915 /proc/interrupts ; sleep 5 ; grep i915 /proc/interrupts
     32: 485 0 PCI-MSI-0000:00:02.0 0-edge i915
     32: 485 0 PCI-MSI-0000:00:02.0 0-edge i915

The GPU compute side remains functional:

    vainfo: VA-API version 1.22 (libva 2.22.0)
    vainfo: Driver version: Intel iHD driver for Intel(R) Gen Graphics - 25.2.6
      VAProfileH264High : VAEntrypointEncSlice  (etc.)
    /lib/dri: card0, renderD128 present

5. WHAT WAS RULED OUT
---------------------

  * Wrong GPU option ROM
      Test:   ADL-N_TWL_GOPv21_igd.rom, gen12_gop.rom, and no rom= at all
      Result: identical fault

  * Display power states
      Test:   i915.enable_dc=0 i915.enable_psr=0 i915.enable_fbc=0
      Result: identical fault

  * DMA beyond 4 GB guest RAM
      Test:   guest memory reduced from 5 GB to 3 GB (below 4 GB)
      Result: identical fault

  * Kodi-specific atomic path
      Test:   raw writes to /dev/fb0
      Result: same commit timeouts

  * EDID / cable / connection
      Test:   connector, EDID and mode all report correct values
      Result: excluded as a cause

  * MSI not enabled
      Test:   lspci -vv shows MSI Enable+; boot-time IRQs are counted
      Result: MSI works for boot, not for display events

  * IOMMU / interrupt remapping
      Test:   hw.vmm.iommu.enable=1
      Result: enabled, fault persists

  * OpRegion / VBT missing
      Test:   OpRegion 8192 B is delivered; VBT at offset 0x400 is zeroes
      Result: suspicious, see below

The bhyve binary on FreeBSD 15.1 carries the following messages, which are
directly relevant to the fw_cfg OpRegion/VBT handover:

    Warning: Unable to reuse host address of OpRegion. GPU passthrough might
not work properly.
    Warning: Unable to reuse host address of VBT. GPU passthrough might not
work properly.

The vt(4) console shows that the pipeline itself works for simple surfaces:
the boot splash surfaces (8-bit, linear) are displayed, and the failure only
starts with the 10-bit tiled GUI plane.

6. EXPECTED RESULT
------------------

Page flips complete, i915 receives vblank/display interrupts, and the GUI is
scanned out on HDMI.

7. ACTUAL RESULT
----------------

The display engine faults on the plane surface fetch, no vblank interrupts are
delivered, and the screen stays black after the splash.

8. ADDITIONAL INFORMATION
-------------------------

  * Kodi 21 on GBM selects a 10-bit GUI plane: CDRMUtils::{} - using 10bit gui
    plane {} with modifier 0x100000000000008 (XR30), which is the plane that
    faults. There is no user setting to force an 8-bit GUI plane on Linux; the
    videoscreen.10bitsurfaces setting does not affect GUI plane selection.

  * LibreELEC 11 (Kodi 20) and the LibreELEC 13 nightly images hang
    immediately after loading the kernel in this VM, with and without the GPU
    passthrough; LibreELEC 12.2 boots normally. This prevented an A/B
    comparison against Kodi 20's legacy (primary-plane-only) GBM path.

  * A reproducer VM and its configuration can be provided on request. The
    minimal reproducer is listed as step 6) above; full guest dmesg and
    i915_display_info output can be attached if useful.

-- 
You are receiving this mail because:
You are the assignee for the bug.