Re: GPU Passthrough on FreeBSD 14.3(AMD Radeon RX 6700 XT and Debian Linux 12.11)

On 8/29/25 1:07 PM, Wouter Snels wrote:
> put something like this in /boot/loader.conf, the numbers below depend 
> on what pci slot your gpu is.
>
> #pptdevs="179/0/0 179/0/1"
That portion of the configuration has already been applied on my end
--

Below is the complete list of arguments passed to bhyve.
> /usr/sbin/bhyve -H -P -A -S -c 4 -m 4096M \
> -s 0,hostbridge \
> -s 1:0,passthru,12/0/0,rom=rx6700xt.rom \
> -s 1:1,passthru,12/0/1 \
> -s 2,nvme,debian.img \
> -s 3,virtio-net,tap0 \
> -s 4,ahci-cd,debian.iso \
> -s 29,fbuf,tcp=0.0.0.0:5900,w=1920,h=1080 -s 30,xhci,tablet \
> -s 31,lpc \
> -l com1,/dev/nmdm2A \
> -l bootrom,BHYVE_UEFI_CODE.fd,BHYVE_UEFI_VARS.fd,fwcfg=qemu \
> debian_vm

The following VM snippets ensures the GPU passthrough to the VM is 
functioning as expected.

> lspci -D | grep -i amd
> 0000:00:01.0 VGA compatible controller: Advanced Micro Devices, Inc. 
> [AMD/ATI] Navi 22 [Radeon RX 6700/6700 XT/6750 XT / 6800M/6850M XT] 
> (rev c1)
> 0000:00:01.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Navi 
> 21/23 HDMI/DP Audio Controller




The GPU resets as soon as a computation task is executed.

> Aug 29 14:38:32 docker kernel: amdgpu: qcm fence wait loop timeout expired
> Aug 29 14:38:32 docker kernel: amdgpu: The cp might be in an 
> unrecoverable state due to an unsuccessful queues preemption
> Aug 29 14:38:32 docker kernel: amdgpu: Pasid 0x8003 DQM create queue 
> type 0 failed. ret -62
> Aug 29 14:38:32 docker kernel: amdgpu 0000:00:01.0: amdgpu: GPU reset 
> begin!
> Aug 29 14:38:32 docker kernel: amdgpu: Failed to suspend process 0x8003
> Aug 29 14:38:32 docker kernel: amdgpu: Failed to suspend process 0x8002
> Aug 29 14:38:33 docker kernel: amdgpu 0000:00:01.0: amdgpu: free PSP 
> TMR buffer
> Aug 29 14:38:33 docker kernel: amdgpu 0000:00:01.0: amdgpu: MODE1 reset
> Aug 29 14:38:33 docker kernel: amdgpu 0000:00:01.0: amdgpu: GPU mode1 
> reset
> Aug 29 14:38:33 docker kernel: amdgpu 0000:00:01.0: amdgpu: GPU smu 
> mode1 reset
> Aug 29 14:38:48 docker kernel: amdgpu 0000:00:01.0: amdgpu: GPU reset 
> succeeded, trying to resume
> Aug 29 14:38:48 docker kernel: [drm] PCIE GART of 512M enabled (table 
> at 0x0000008000000000).
> Aug 29 14:38:48 docker kernel: [drm] VRAM is lost due to GPU reset!
> Aug 29 14:38:48 docker kernel: [drm] PSP is resuming...
> Aug 29 14:38:49 docker kernel: [drm:psp_hw_start [amdgpu]] *ERROR* PSP 
> create ring failed!
> Aug 29 14:38:49 docker kernel: [drm:psp_resume [amdgpu]] *ERROR* PSP 
> resume failed
> Aug 29 14:38:49 docker kernel: [drm:amdgpu_device_fw_loading [amdgpu]] 
> *ERROR* resume of IP block <psp> failed -62
> Aug 29 14:38:49 docker kernel: amdgpu 0000:00:01.0: amdgpu: GPU 
> reset(1) failed
> Aug 29 14:38:58 docker kernel: amdgpu: qcm fence wait loop timeout expired
> Aug 29 14:38:58 docker kernel: amdgpu: The cp might be in an 
> unrecoverable state due to an unsuccessful queues preemption
> Aug 29 14:38:58 docker kernel: amdgpu 0000:00:01.0: amdgpu: GPU reset 
> end with ret = -62
> Aug 29 14:38:58 docker kernel: amdgpu 0000:00:01.0: amdgpu: GPU reset 
> begin!
> Aug 29 14:38:59 docker kernel: amdgpu 0000:00:01.0: amdgpu: Failed to 
> disallow df cstate

The vm is still up and running however the GPUs capabilities are lost.

Regards,
Petru