Re: Feb 19 meeting actions, discussion, notes, and recording

On Tue, Mar 4, 2025 at 9:47 PM Ed Maste <emaste@freebsd.org> wrote:
>
> The Foundation is investigating funding some libvirt work. There are
> no specific features in scope to start. We're looking at general
> maintenance and maintainability: surveying open bugs, helping to
> improve or stand up CI testing, etc.
>
> Can you describe your initial experience with virt-manager/libvirt/bhyve?

Let me dump all our virtualization-related movements, I hope a broader
picture wouldn't hurt anyone.

* We started with using bhyve to run a Windows guest completely in the
background and are accessing it via FreeRDP "seamless" mode for most
apps and Moonlight client for games.
We achieved moderate success with this setup -  it is fast in terms of
rendering, Windows windows looks pretty "native" in the seamless mode
and the passed-through dGPU works like a charm. We even played Mortal
Kombat 11 while capturing the video.
* The serious drawback we're currently facing on the bhyve side is the
suspend/resume functionality (aka snapshots). If we manage to find
enough resources, we're hoping to bring Vitaly Gusev to work on this,
since he was developing this feature earlier.
* Another problem is having an ability to configure bhyve networking
in the same way as VirtualBox' "NAT" option works - no network
interface or any configuration should be required from the host side.
This is achieved with libslirp and is being discussed here:
https://reviews.freebsd.org/D42510#inline-292152
* Having bhyve running prevents other hypervisors from running, like
VirtualBox. This means that our user would need a neat GUI to create
and run his own VMs.
* bhyvemgr looks nice, but it resorts to sudo, which is a too
simplistic approach for the prodcution use. The canonical approach for
such tasks is to have a daemon running as root to actually perform
privileged operations and then have a RPC protocol for non-root client
applications to talk to it. The contemporary desktop environments are
using DBus as an RPC bridge and the Polkit framework to define and
manage policies for actions requiring escalated privileges.
* We at $WORK also have somewhat strange and murky requirements for a
VM to have multiple users with various roles, like "VM Developer", "VM
Administrator", etc. Such requirements can't be fulfilled without a
centralized daemon that performs mandatory control and all other
security checking stuff.
* This is where libvirt comes into play - it is an infrastructure that
packs a lot of things. Not all of them are useful to us - for instance
we don't need to abstract from the underlying backend and always plan
to use bhyve. But useful things are the privileged daemon
implementation, a D-Bus API and a GUI. The GUI part, virt-manager,
looks too "advanced" for a regular user, so we'll probably end up
writing our own alternative frontend that will hide the unnecessary
complexity of this beast.
* We already put some patches for libvirt to upstream:
https://lists.libvirt.org/archives/list/devel@lists.libvirt.org/thread/JOZ26QFIU4O3YTDMV7E5JYMGDYC7DZ4H/
This will allow configuring passthrough devices via libvirt. Thanks
Roman who provided a review and pushed some relevant bits already.
* Finally, there is KDE Karton project
https://invent.kde.org/sitter/karton/ which looks much more like what
we need, but it is still in an extremely early stage.

To sum this up, our general direction is to
- improve bhyve itself
- improve bhyve support in libvirt
- come up with a GUI frontend: either writing it from scratch, or
chiming into Karton's development
- improve virt-manager on occasion since we'll probably use it as a
"reference" frontend implementation that gives a broader access to
libvirt's APIs