bhyve and vfs.zfs.arc_max, and zfs tuning for a hypervisor

[Victor Sudakov]

Patrick M. Hausen wrote:
> 
> > 1. Does ARC actually cache zfs volumes (not files/datasets)?
> 
> Yes it does.
> 
> > 2. If ARC does cache volumes, does this cache make sense on a hypervisor,
> > because guest OSes will probably have their own disk cache anyway.
> 
> IMHO not much, because the guest OS is relying on the fact that when
> it writes it’s own cached data out to „disk“, it will be committed to
> stable storage.

This is an important point.

> > 3. Would it make sense to limit vfs.zfs.arc_max to 1/8 or even less of
> > total RAM, so that most RAM is available to guest machines?
> 
> Yes if you build your own solution on plain FreeBSD. No if you are running
> FreeNAS which already tries to autotune the ARC size according to the
> memory committed to VMs.
> 
> > 4. What other zfs tuning measures can you suggest for a bhyve
> > hypervisor?
> 
> e.g.
> 	zfs set sync=always zfs/vm
> 
> if zfs/vm is the dataset under which you create the ZVOLs for your emulated
> disks.

Well, bhyve already has an option for this:

The block-device-options are:

nocache   Open the file with O_DIRECT.
direct    Open the file using O_SYNC.
ro        Force the file to be opened read-only.

I think something like 
"-s 4:0,virtio-blk,/dev/zvol/zroot/vm/mail/disk0,direct"
would do the same?

> 
> I’m using this for all my VM „disks“ and have added a 16 GB SLOG device
> to my spinning disk pool - seems to work great. This is on a home system.

Is SLOG also used by zfs volumes?

-- 
Victor Sudakov,  VAS4-RIPE, VAS47-RIPN
2:5005/49 at fidonet http://vas.tomsk.ru/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 455 bytes
Desc: not available
URL: <http://lists.freebsd.org/pipermail/freebsd-virtualization/attachments/20190320/d324d5f7/attachment.sig>