libvirt improvements for bhyve

[Roman Bogorodskiy]

  Craig Rodrigues wrote:

> Roman,
> 
> I am using bhyve a lot these days, but
> I find that I need to write scripts on top of
> bhyveload / bhyve in order to work with VM's.
> 
> I would rather use libvirt, because there are many 3rd party
> programs to work with VM's that are written for libvirt (especially for
> KVM).
> 
> How hard would it be to implement the following:
> 
> (1)  The last time I tried libvirt 1.2.7, it did not seem to properly keep
> track
>        if a VM was powered on or shutdown.  If I started a VM with libvirt,
>       then did "shutdown -r now", I couldn't seem to restart the VM, unless
>       I destroyed the VM in libvirt, and restarted.
>       This is very inconvenient.
> 
>       If you see the latest version of vmrun.sh (
> https://svnweb.freebsd.org/base/head/share/examples/bhyve/vmrun.sh?view=markup
> )
>       you will see that I added some comments to tell when the VM has
>       been reset, powered off, etc. by looking at the exit status of bhyve.
> 
>       Can we add this logic to libvirt, so that it can properly tell if
>       a VM is up or not?

Frankly speaking, it's a hard topic for me and currently I don't have a
complete idea how to design that. The thing I'm worried about is that as
far as I understand that exit codes are sort of temporary solution and
will be dropped in the future. Also, it seems the behaviour is changing
already, e.g. IIRC on -CURRENT the bhyve proccess doesn't exit on
reboots (unlike in 10-R). My goal is to support 10-R and I'm not sure
how to probe if exit codes are available or not (and how to probe if
e.g. if new exit code will be added in some future version).

On the other hand, I think I need to check cases when bhyve process
vanishes away and probably set VM status to something like 'shutdown /
error' or something like that.

Also, I think I need to support 'bhyvectl --force-reset' if that's
available.

I guess that should be safe for all versions of bhyve and I'll add it to
my todo list.

> (2)  "bhyveload -e" allows specifying environment variables to loader.
>       Can we have a way to specify loader environment variables,
>       in the libvirt XML config?
> 
> (3)  "grub-bhyve" is needed to boot VM's for Linux.
>       Can we specify an option to choose between "bhyveload" and
> "grub-bhyve"?

For 2-3 I think bootloader should be honored from the XML file:

http://libvirt.org/formatdomain.html#elementsOSBootloader

I'll add it to my todo list as well.

> (4)  bhyve has many options for specifying which PCI slots to use,
>       and also PCI passthru ( https://wiki.freebsd.org/bhyve/pci_passthru )
>       Can this be specified in libvirt?

Currently you can specify PCI slot and function by using 'address' tag
in the device element (e.g. disk or interface), for example:

    <interface type='bridge'>
      <mac address='52:54:00:f6:41:4e'/>
      <source bridge='virbr0'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </interface>

PCI passthrough is not supported as I've never looked at that, will need
to take a look.

> 
> For me, without (1), libvirt is mostly unusable for bhyve.  (2) - (4) would
> be
> nice to have.  However, if we could fix libvirt with these things,
> then a lot of things I do now in scripts, I could do in libvirt.
> I would much rather use libvirt, because then I could take advantage of
> a lot of libvirt software targeted towards KVM.
> 
> Thanks.
> --
> Craig

Roman Bogorodskiy
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 488 bytes
Desc: not available
URL: <http://lists.freebsd.org/pipermail/freebsd-virtualization/attachments/20141011/57904f49/attachment.sig>