Re: GPU Passthrough on FreeBSD 14.3 (AMD Radeon RX 6900 XT and Windows 10 Pro)

I'm continuing my investigation of why I don't see the "Shutting down" message when I run "shutdown -r now" but I do if I do "service vm_gaming stop". I'm wondering if it's actually gracefully shutting down but I just don't see my monitor update because things are happening quickly. There was 1-2 times I did get the "shutdown -r now" to cause my vm to show me that screen, but it doesn't show it to me every time. So far I haven't gotten any Windows disk corruption or anything, but just not confident about the graceful shutdown.

I implemented a "status" function to see if maybe it's some early termination logic that's causing it not to run (or exit too early), and I've also replaced my "sleep 10" with a proper while loop that will check the existence of the child pid. Once the child pid no longer exists, then execution continues in the stop function.

The script currently looks like this:

#!/bin/sh

# PROVIDE: vm_gaming
# REQUIRE: LOGIN
# KEYWORD: nojail shutdown

. /etc/rc.subr

name=vm_gaming
rcvar=vm_gaming_enable

vm_path="/atlantis/vms/gaming"
vm_name="gaming"

start_cmd="${name}_start"
stop_cmd="${name}_stop"
restart_cmd="${name}_restart"
status_cmd="${name}_status"

pidfile="/var/run/${name}.child.pid"

vm_gaming_start()
{
	echo "starting gaming vm ..."
	daemon \
		-o /var/log/${name}.log \
		-p "${pidfile}" \
		bhyve -AHPSw -c sockets=1,cores=16,threads=1 -m 32G \
		-s 0,hostbridge \
		-s 1,nvme,${vm_path}/disk0.img \
		-s 3:0,passthru,3/0/0 \
		-s 3:1,passthru,3/0/1 \
		-s 13:0,passthru,13/0/0 \
		-s 30,xhci,tablet \
		-s 31,lpc \
		-l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd,fwcfg=qemu \
		-o console=stdio \
		${vm_name}
}

vm_gaming_stop()
{
	echo "stopping gaming vm ..."

	# Send SIGTERM twice to make sure Windows listens to
	# the ACPI shutdown signal.
	pid="$(cat ${pidfile})"
	kill ${pid}
	kill ${pid}

	# Wait a bit for the guest to shutdown properly before
	# we continue shutting down the host.
	is_pid_running="$(ps ${pid} -o pid=)"
	while [ ! -z "${is_pid_running}" ]; do
		is_pid_running="$(ps ${pid} -o pid=)"
	done

	bhyvectl --vm=${vm_name} --destroy

	echo "gaming vm stopped!"
}

vm_gaming_restart()
{
	# NOTE: AMD users will most likely experience the famous
	# AMD Hardware Reset Bug. This means that after you reboot
	# the guest, you most likely won't have video out. If this
	# happens, you'll need to restart the host. Sometimes this
	# command will work for AMD users though. So you can try a
	# restart and see if it works.
	vm_gaming_stop
	vm_gaming_start
}

vm_gaming_status()
{
	if [ -f "${pidfile}" ]; then
		echo "$(cat ${pidfile})"
	fi
}

load_rc_config $name

: ${vm_gaming_enable:="NO"}

run_rc_command "$1"