Suspend/resume question with Ubuntu guest question

From: Roman Bogorodskiy <novel_at_freebsd.org>
Date: Thu, 09 Oct 2025 16:28:17 UTC
Hi,

I'm trying to get suspend/resume working without success so far.

My setup is:

 - 16.0-CURRENT #0 main-n280778-f45608124286: Sun Oct  5 10:30:44 CEST 2025
 - Ubuntu 25.10 guest

I run the guest with:

bhyve -c 2 -m 4096 -A -I -u -H -P -s 0:0,hostbridge -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
  -s 2:0,xhci,tablet -s 1:0,lpc \
  -s 3:0,ahci,hd:/data/img/ubuntu2510.img \
  -s 4:0,virtio-net,tap5,mac=52:54:00:02:93:f1 \
  -s 5:0,fbuf,tcp=127.0.0.1:5955 -l com1,/dev/nmdm200A ubuntu2510

Guest starts fine and works as expected. I suspend it using:

bhyvectl --vm=ubuntu2510 --suspend=/var/run/libvirt/bhyve/ubuntu2510

That also goes well, bhyve terminates, VNC and network connections drop,
no instance in /dev/vmm, and files are there:

# du -h /var/run/libvirt/bhyve/ubuntu2510*
4,0G    /var/run/libvirt/bhyve/ubuntu2510
 32M    /var/run/libvirt/bhyve/ubuntu2510.kern
4,0K    /var/run/libvirt/bhyve/ubuntu2510.meta

Then I resume the guesting using the following command:

bhyve -r /var/run/libvirt/bhyve/ubuntu2510 \
 -c 2 -m 4096 -A -I -u -H -P -s 0:0,hostbridge -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
 -s 2:0,xhci,tablet -s 1:0,lpc -s 3:0,ahci,hd:/data/img/ubuntu2510.img \
 -s 4:0,virtio-net,tap5,mac=52:54:00:02:93:f1 \
 -s 5:0,fbuf,tcp=127.0.0.1:5955 -l com1,/dev/nmdm200A ubuntu2510

Here issues begin: I can see the bhyve process running and there is an
entry in /dev/vmm for this guest, but it does not respond to network
connections. I can connect via VNC but it just shows a static image of
the screen at the point when the guest was suspended.

I don't seem to see any related error messages either, except probably:

vatpit_snapshot: snapshot restore does not reset timers!

In /var/log/messages for both suspend and resume events.

Another interesting thing: if I destroy the guest now, it prints:

fbuf frame buffer base: 0x1d60d9600000 [sz 33554432]
Pausing pci devs...
Restoring vm mem...

Restoring pci devs...
Restoring kernel structs...
Resuming pci devs...

And then exits.

Am I doing something wrong/using incompatible device configuration, or
is that a bug?

PS Also checked with a FreeBSD guest: same result.

Thanks,
Roman