backups of bhyve images

[Alfred Perlstein]

On Dec 8, 2014, at 8:33 AM, John wrote:

> Hello list,
> 
> I have a few questions about creating backups to be stored offsite.
> 
> If a guest is running, can I compress the image without it becoming 
> inconsistent? If not, can it be copied without it becoming inconsistent?  
> By inconsistent, I mean will I see weird effects and broken files if the 
> backup is restored? Previously I've shut the VM down to avoid this, 
> before archiving.
> 
> I have each image on its own (external to the image) ZFS filesystem.  
> Internally the image is using ufs if freebsd, ext3fs if linux.  Would 
> using some ZFS method of duplication be better? In this case, would the 
> image become inconsistent?
> 
> Basically, what I want to do is to run accurate backups without shutting 
> down and restarting the VM. Is this possible? If it isn't, I think the 
> only alternative is to make a script that shuts the vm down, copies it, 
> restarts the vm then runs its compression and backup-over-ssh routine.

[[ adding fs at freebsd.org in case I'm wrong ]]

If you are using UFS internally to the VMs then you'll need to send a snapshot that is consistent.

If you are just copying the files out from under a running vm you are going to get spaghettios for a filesystem if you try to recover as you need a true point in time snapshot.

I think a few better options would be:
1) Inside the VM create a UFS snapshot then dump that externally using tools.
2) Create the UFS snapshot, then make sure that the file/vzol is snapshotted using zfs.
3) Just snapshot the underlying zvol you've made the UFS image on and send that (you'll get a dirty FS on restore, but it *should* be recoverable with a simple fsck)  
4) Use zfs internally to the vm and send/receive the internal zfs.

option 3 is the least safe imo as you can wind up with filesystem "angry".
in case 1 and 2 you'll have UFS snapshots that should be "OK" to restore from.
in case 4 you are also doing snapshot, but you switch to ZFS.

-Alfred