Can a host OS user process create a zillion BHyVe VM:s and microcontrol them?

[Peter Grehan]

> Right, so a VT-x takes time - if we're talking 2500 cycles per
> roundtrip, then you get approx one million of them per second - fair
> enough.

  Not much time left for anything else though at that exit rate :(

> As you point out the VM is launched absolutely bare.
>
> So then, strategies would need to be devised for how to run the code.
>
> I guess this would range all the way from running a barebones memory
> management and pthreads implemenation (both those exist at least for
> amd64, if I got it right, not sure how up to date they are though),
>
> ..to running a minimal BSD/Linux kernel within the VM for maxing
> compatibility.
>
> Perhaps some neat memory dumping drick could be applied to zero the
> BSD/Linux kernel boot time, i.e. the kernel doesn't boot but it's just
> loaded from a hibernated memory image.

  Yep, that's possible.

> Okay, so, there are two aspects to this problem now:
>
>   (1) The BHyVe specific parts, about how to use BHyVe's interfaces to
> deliver the virtualization/sandbox aspect, and
>
>   (2) The guest specific parts, as in how run the intended
> libraries/binaries as close to the metal as possible and with as low RAM
> overhead as possible (and CPU overhead) within the sandbox.
>
>
> I guess (2) is off-topic to this mailing list so should be looked into
> separately.
>
> If you have any spontaneous thoughts on that one though, of course feel
> free to share them :)

  There are many examples of small embedded o/s's that could easily be 
run in this environment - a reasonable example is the FreeBSD loader itself.

> And, (1) is on this ML's topic, so I can direct any subsequent questions
> about that to here.
>
> If you have any particular references with regard to header files, man
> page or example program source files, feel free to pass them!

  Not a lot of documentation other than the bhyve source code itself.

  lib/libvmmapi/vmmapi_freebsd.c is an example of how to set up a 64-bit 
environment that the FreeBSD/amd64 kernel can be directly executed from.

 
https://github.com/grehan-freebsd/grub2-bhyve/blob/master/grub-core/kern/emu/bhyve_hostif.c 
sets up a flat 32-bit environment that Linux/OpenBSD/NetBSD kernels 
expect to be invoked in.

  usr.sbin/bhyve/bhyverun.c has the main run loop for bhyve, which 
demonstrates how exits are handled.

later,

Peter.