bhyve and contention

Rodney W. Grimes freebsd-rwg at
Tue Feb 13 18:07:09 UTC 2018

> Hello virtualizations,
> Please can anyone tell me what happens in the following scenarios:

I can speculate on some of what may happen, this may not be exact,
but should give you an idea.

> 1.
> freebsd-11 server, 5x freebsd-11 guests
> The server has 32GB ram installed. it's an i7 so 8 cores. There's 64GB
> swap. Each guest that bhyve loads is with m 8192M, so ram is
> overcommitted by 6GB.

5 x 8G == 40G, 32G ram, so 8G overcommit, not 6.  Further more you
have base sytem memory which includes such things as kernel, page
tables, loaded modules, buffer cache, possibly ZFS arc cache since
you dont mention that.   ZFS arc cache will attempt to grow to
something like 95% of memory, so you can actually be in full 40G
over commit situation on memory.

> What happens to the guests and the server? Do either (or both) crash? Or
> is there something in bhyve that will make memory allocated on a
> best-effort basis?

With 64G of swap space avaliable, and assuming your either not running
ZFS or have limited zfs arc cache to say 2G or 4G the system should
not crash.  You however are in a memory overcommit and things are likely
to be sluggish and may even appear crashed.

Bhyve uses swap backed annonymous vm so the vm system shall apply
best effort to meet the demands.  Swapping vm guest memory pages
though not idea does work, but as I said, things are going to
get very sluggish both on the host and the guest.

> 2.
> In the following context, the server is the same but this time all five
> guests have -c 4 per guest, so bhyve is asking 12 more cores than that
> existing in hardware. Does the guest fail to load, do either guest or
> server crash?

The is core over commit, very common in the virtualization world,
bhyve does its best effort to give the guests cores as needed.
I am typing this on a system with a 4 core CPU (2 real x 2 thread)
that is running 10 bhyve VM's, each just single core.  My host
load average is:
 6:01PM  up 3 days, 16:01, 7 users, load averages: 3.77, 3.34, 3.22
One of those guests is running a stable/10 make world in a memory
contrained situation (512MB).

The host system is responsive and working fine, as are the

I am fine with CPU over commit, but I watch the memory situation
like a hawk.
I only have 12GB of memory, I have restricted ZFS arc cache to 6G
The sums of the memory of all the running VM's is about 2G
This leaves me about 4G of fiddle around room on the host for
occasional VM spin up, or other memory heavy things.

I run all my VM's with the wired memory option.

> thanks,
> -- 
> J.
> _______________________________________________
> freebsd-virtualization at mailing list
> To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe at"

Rod Grimes                                                 rgrimes at

More information about the freebsd-virtualization mailing list