bhyve uses all available memory during IO-intensive operations

[K. Macy]

There was a standards group but now the interfaces used buy the Linux
virtio drivers define the de facto standard. As virtual interfaces go
they're fairly decent. So all we need is a backend.

The one thing FreeBSD doesn't have that I miss is CPU hot plug when running
as a guest - or at least a mechanism to be told to stop running on the APs.
That would make live migration much simpler.


-M



On Sat, Dec 2, 2017 at 12:03 Rodney W. Grimes <
freebsd-rwg at pdx.rh.cn85.dnsmgr.net> wrote:

> > On Fri, Dec 1, 2017 at 20:02 Rodney W. Grimes <
> > freebsd-rwg at pdx.rh.cn85.dnsmgr.net> wrote:
> >
> > > > On 02/12/2017 08:11, Dustin Wenz wrote:
> > > > >
> > > > > The commit history shows that chyves defaults to -S if you are
> > > > > hosting from FreeBSD 10.3 or later. I'm sure they had a reason for
> > > > > doing that, but I don't know what that would be. It seems to an
> > > > > inefficient use of main memory if you need to run a lot of VMs.
> > > >
> > > > It sounds like a reasonable solution to a problem. If host memory is
> > > > full it swaps some out, so a bhyve might have free mem but some
> could be
> > > > swapped out by the host. If the bhyve is out of mem, it's system
> swaps
> > > > to it's disk, so the host swaps it back in so that the bhyve can then
> > > > swap it to its disk...
> > > >
> > > > Wiring bhyve ram might be a reasonable solution as long as the hosts
> > > > physical ram isn't over allocated by bhyve guests.
> > > >
> > > > The best solution would involve a host and guest talking to each
> other
> > > > about used mem, but that would break the whole virtual machine
> illusion.
> > > > At the least it would involve a system telling the hardware what
> memory
> > > > is used and what is not, which just isn't something any system does.
> > > > Maybe that is an idea for the vm guest aware systems of the future.
> > >
> > > Its actually old technology, its called the memory balloon driver,
> > > but bhyve does not have that functionality, yet.
> > >
> > >
> >
> > The virtio ballon driver is already there. Implementing a kernel backend
> > for it would be trivial. In kernel virtio-net and virtio-p9fs backends
> are
> > already well underway.
>
> Don't you also need guest front ends for each OS?  That is the hard part
> from what I have seen of all the memory over commit stuff.  Especially when
> you get to the part of "this page of memory in this guest is the exact
> same thing as that page of memory in that guest".
>
> But if we had the backend working, and just the FreeBSD guest frontend
> it would be a big win for many of us using bhyve with quantities of
> FreeBSD guests.
>
> Are we compativle enough we can use the KVM guest balloning extensions?
>
> --
> Rod Grimes
> rgrimes at freebsd.org
>