bhyve and contention

Shane Ambler FreeBSD at ShaneWare.Biz
Wed Feb 14 03:23:32 UTC 2018

On 14/02/2018 10:18, Peter Grehan wrote:
>>> 2.
>>> In the following context, the server is the same but this time all five
>>> guests have -c 4 per guest, so bhyve is asking 12 more cores than that
>>> existing in hardware. Does the guest fail to load, do either guest or
>>> server crash?
>> The is core over commit, very common in the virtualization world,
>> bhyve does its best effort to give the guests cores as needed.
>  To add to what Rod said - bhyve uses a thread for each vCPU. It's up to
> the FreeBSD scheduler to determine where/when these threads run.
>  It is possible for a guest to fail, for example if a spinlock times out
> due to vCPUs not being able to run to release a lock. This is a general
> problem with virtualization and can occur even on VMWare ESXi with heavy
> oversubscription.

There are  some bhyve options you need to check -

You want to use the -H option which prevents the guest keeping 100% cpu.

Be sure not to pass the -S option to bhyve as it wires the guest memory.
This was seen recently as sysutils/chyves uses it by default.

If you use zfs, you may want to set the arc_max.

One thing that I have noticed, as a guest allocates ram, the host
allocates ram to its bhyve instance, but when the guest releases it, the
host is unaware of the ram now being free. Apparently there is work on
getting the guest to notify the host of released ram. Until that is
available you may want to restart guests that temporarily allocate large
amounts of ram.

FreeBSD - the place to B...Sharing Devices

Shane Ambler

More information about the freebsd-virtualization mailing list