EBS snapshot backups from a FreeBSD zfs file system: zpool freeze?

[Kevin Day]

On Jul 3, 2013, at 12:50 AM, Jeremy Chadwick <jdc at koitsu.org> wrote:

> On Wed, Jul 03, 2013 at 02:08:13PM +1200, Berend de Boer wrote:
>> 
>> For starters, I suppose very few people are using FreeBSD on AWS, so
>> "most of us" don't have a choice :-)
>> 
>> But this might simply be my understanding: what if I want to use the
>> EBS snapshot of the 5 disks I've taken and attach them to another
>> machine, and mount it?
>> 
>> But perhaps you don't know what an EBS snapshot is? It's not a backup
>> of your file system, it's a hardware-based backup of a disk at a
>> single point in time.
>> 
>> I.e. with EBS I can take a snapshot of 5 1TB, create new disks from
>> it, and attach it to another machine. In SECONDS.
> 
> Okay, I think I understand what you're asking.  Please correct me:
> 
> It sounds to me like the Linux OS images on AWS have utilities or the
> capability to create EBS images that are snapshots of the "virtual
> disks" that make up the AWS system, and that you can transfer these
> to another Linux AWS machine and mount the EBS images, and that this is
> being done within Linux itself.
> 
> Correct?
> 
> If so -- then what you're wanting to ask is: "does FreeBSD have support
> for EBS images?"  (Meaning this has nothing to do with ZFS)  I get the
> impression an EBS image is a proprietary Amazon thing, so you would need
> to ask Amazon if they have the same utilities for FreeBSD, or ask the
> individual(s) responsible for the FreeBSD AWS images if there are such
> tools.  Again: nothing to do with ZFS.

Not quite, we run into something similar with VMware ESXi, so let me try to rephrase it.

ESXi provides block storage to virtual machines that appear to be traditional disks. ESXi has no real knowledge of the contents, files, it's just dumb block storage. This is like EBS, the volumes appear to be normal disks to the virtual machine.

But, the backend storage system allows you to take snapshots of the virtual disks. You can then roll back to an earlier snapshot, or make copies of the snapshots for other VMs to use. The problem is that unless you quiesce the filesystem before making the snapshot, the contents are potentially "dirty". It could be mid-write, or in the middle of doing something that would be bad to leave that way. If you're unlucky and make a snapshot during one of those inconsistent periods, if you were to rollback or clone that snapshot you'd get fsck complaining (for UFS) or ZFS may need to do some repairing, etc. This is much less an issue with ZFS than other filesystems, but it's still possible to think you've written something, try to trigger a snapshot, and it's not actually committed to disk yet.

Linux has an ioctl(?) you can call that says "Quiesce this filesystem", where all dirty buffers are flushed and (i believe), writes are blocked, and the filesystem is temporarily marked clean. You then trigger the backend storage to make a filesystem, then you unfreeze the filesystem so that normal I/O can continue.

The closest thing we can do in FreeBSD is to unmount the filesystem, take the snapshot, and remount. This has the side effect of closing all open files, so it's not really an alternative. 

The other option is to not freeze the filesystem before taking the snapshot, but again you risk leaving things in an inconsistent state, and/or the last few writes you think you made didn't actually get committed to disk yet. For automated systems that create then clone filesystems for new VMs, this can be a big problem. At best, you're going to get a warning that the filesystem wasn't cleanly unmounted.


-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4891 bytes
Desc: not available
URL: <http://lists.freebsd.org/pipermail/freebsd-fs/attachments/20130703/c9906e5b/attachment.bin>