[Bug 300031] bhyve iGPU passthrough on Alder Lake-N: display engine faults on 10-bit/tiled GUI planes, i915 never gets vblank interrupts

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=300031

--- Comment #3 from wanpengqian@gmail.com ---
Hi Michael,

We have run into both of these symptoms while working on iGPU passthrough
for keelOS (https://keelos.dev, a FreeBSD 15 based hypervisor built on
bhyve), and we have a patch for each. Both are against releng/15.1, so they
apply to your system as they are (patch -p1 in /usr/src). They are
attached here; we will also put them on Phabricator as soon as
reviews.freebsd.org is back from maintenance and post the links. It
would help a lot if you could try them.

1) Plane faults (attachment 0001, bhyve)

On a Comet Lake iGPU (Lenovo P340) we saw "[PLANE:..] fault" on every
frame, and the host's VT-d logged reads by the GPU from addresses far
outside the guest's memory. The cause: the firmware fills only the global
GTT entries it uses, the rest keep stale host addresses, and i915 in a
guest does not overwrite them. The display engine prefetches past the end
of the surface through those entries. The patch points every GTT entry at
a page of the guest's own memory before the guest starts, much as QEMU
clears the table. The faults have been gone on that machine since.

2) flip_done / commit wait timeouts (attachment 0002, kernel)

On Alder Lake-S (UHD 770) a Linux guest waited 10 s for every flip, with
"flip_done timed out" and "DSB n timed out waiting for idle". The i915
interrupt counter stopping does not mean interrupts are lost: in our
traces the vblank interrupt was masked while nobody was waiting for it,
and it was delivered when it was unmasked. The stall was the Display
State Buffer (Gen12+), which the display engine reads by DMA without
snooping the CPU caches. Our explanation: bhyve's EPT entries ignore
the guest's PAT, so the WC buffers the driver writes for the display are
really write-back, and the display engine reads stale memory. The patch adds a
loader
tunable, vm.pmap.ept_guest_pat=1, that lets the guest's PAT count. (Our
own build turns this on per VM for guests with passthrough devices; the
tunable is only for testing.) We have not yet confirmed this one on
hardware ourselves.

A quick check before rebuilding anything: boot the guest with
i915.enable_dsb=0. If the GUI then comes up, it is the DSB problem.

To test:
  - 0001: rebuild and install usr.sbin/bhyve.
  - 0002: rebuild the kernel, add vm.pmap.ept_guest_pat=1 to
    /boot/loader.conf, reboot (sysctl vm.pmap.ept_guest_pat should be 1).

Please report the guest's dmesg (plane faults, flip_done timeouts, DSB
lines) with 0001 alone, and with 0001 + 0002 without i915.enable_dsb=0.
If they fix it, we will rebase them onto main for commit.

Thanks for the detailed report.

Wanpeng Qian

-- 
You are receiving this mail because:
You are the assignee for the bug.