bhyve: zvols for guest disk - yes or no?

[Patrick M. Hausen]

Hi, all,

> Am 17.11.2016 um 11:16 schrieb Jan Bramkamp <crest at rlwinm.de>:
> Bhyve doesn't support page deduplication but it doesn't wire down guest memory unless you asked it to, but you have to wire down guest memory to use PCI passthrough. If I were picking VM hosts today I would go with LGA 2011 v3 boards having least eight DDR4 slots per socket. Maybe use some nice >= 2TB NVMe SSDs and suddenly your limited by CPU cycles and storage space instead of IOPS and RAM.

Perhaps I'll describe the intention of this project. All your suggestions
are great for a heavy virtualisation platform. In our case we just want
a cheap, yet "reliable enough" platform to host a handful of rocket.chat
instances.

While rocket.chat claims FreeBSD support that's way behind compared
to running the product on Linux. We found Ubuntu to be a good choice
in this case.

Unfortunately the application's architecture doesn't easily fit running
multiple instances on one kernel/OS. The absence of current FreeBSD
support rules out jails. So a full-fledged hypervisor is called for.

And we will use a system with two mirrored SATA disks to run no more
than 10 Ubuntu guests with at most 4 GB each and 100 GB of guest
disk space. 1 or 2 TB of spinning rust in a mirror, that's all.
At least rocket.chat is not performance hungry.

The only issue I saw is RAM (CPU supports at most 32 GB) and the
ZVOL "issue" I initially asked about. We will now provision them "sparse"
with a small block size ...

> An other thing I learned the hard way is that ZVOL are set in stone at the ZVOL creation. You have to (cam)dd everything to change the block size. The default ZVOL block size is 8K which isn't wrong but your guests need to align their file systems (and swap) correctly or you'll suffer from write amplification. And ZFS RAID-Z really sucks for such small block sizes. Use mirrored VDEVs in your pools or you will suffer from massive metadata overhead and disappointing IOPS.

I attended Kirk's kernel class on last EuroBSDCon ;-) You are essentially
preaching to the choir - but for the list archives and later searches sake,
your work will probably do somebody else some good.

Thanks again,
Patrick
-- 
punkt.de GmbH * Kaiserallee 13a * 76133 Karlsruhe
Tel. 0721 9109 0 * Fax 0721 9109 100
info at punkt.de       http://www.punkt.de
Gf: Jürgen Egeling      AG Mannheim 108285