Options for zfs inside a VM backed by zfs on the host

[Marcus Reid]

On Wed, Aug 26, 2015 at 05:25:52PM -0400, Vick Khera wrote:
> I'm running FreeBSD inside a VM that is providing the virtual disks backed
> by several ZFS zvols on the host. I want to run ZFS on the VM itself too
> for simplified management and backup purposes.
> 
> The question I have is on the VM guest, do I really need to run a raid-z or
> mirror or can I just use a single virtual disk (or even a stripe)? Given
> that the underlying storage for the virtual disk is a zvol on a raid-z
> there should not really be too much worry for data corruption, I would
> think. It would be equivalent to using a hardware raid for each component
> of my zfs pool.
> 
> Opinions? Preferably well-reasoned ones. :)

This is a frustrating situation, because none of the options that I can
think of look particularly appealing.  Single-vdev pools would be the
best option, your redundancy is already taken care of by the host's
pool.  The overhead of checksumming, etc. twice is probably not super
bad.  However, having the ARC eating up lots of memory twice seems
pretty bletcherous.  You can probably do some tuning to reduce that, but
I never liked tuning the ARC much.

All the nice features ZFS brings to the table is hard to give up once
you get used to having them around, so I understand your quandry.

Marcus