bhyve VM drive size limit?

Jason Tubnor jason at
Sun Sep 3 03:52:00 UTC 2017

Hi Ruben,

On 3 September 2017 at 00:52, Ruben <mail at> wrote:

> Perhaps slightly off-topic, but might you have any recommendations
> regarding those blocksizes?

I use 8k zvol block sizes.  This seems to work consistently for me with
OpenBSD and Windows Server 2016 guests.  While you could probably go
larger, it really depends on your use case and how the data intends to move
around in the guest.  Some people have had performance issues with 8k
blocks, however, I am yet to experience this.  Chyves by default used 512
byte block sizes, this is sub-optimal for a guest with > 9GB storage needs,
so I change the default settings back to 8k.

> I'm not really that well versed in blocksizes / filesystems and am often
> wondering where my 5TB zvol (bhyves into an ext4 volume for an ubuntu
> guest) exactly disappears into :P

For the above, I have experienced where block sizes were defined at 512
bytes, the associated block checksums would eat up the storage, just as
quick as the data being committed to the zvol (even more so as the
snapshots pile up).  Increasing the zvol block size increased the storage
to checksum ratio, reducing how much the checksum overhead had an affect on
the storage.  Keep in mind, this is with a zmirror, so mileage may vary
with raidz or raidz2 setups.  Another thing to keep in mind, as you
increase the block size, this can have performance impacts on the guest as
a 4k update in the guest filesystem, would but upwards pressure on array
performance if you used say 32-64k blocks (rewriting > 32x the
data+checksum calcs).

I suggest trying different sizes for the same workload you plan to commit
to the guest storage to see which has the best storage to performance ratio.



P.S.  In the above, I wouldn't commit the 5TB to the guest in one hit.
Either grow it out as you need it or add additional zvols to your guest as

More information about the freebsd-virtualization mailing list