EBS snapshot backups from a FreeBSD zfs file system: zpool freeze?

[Markus Gebert]

On 07.07.2013, at 23:53, Berend de Boer <berend at pobox.com> wrote:

>>>>>> "Markus" == Markus Gebert <markus.gebert at hostpoint.ch> writes:
> 
>    Markus> But taking a zfs snapshot is an atomic operation. Why not
>    Markus> use that? For example:
> 
>    Markus> 1. snapshot the zfs at the same point in time you'd issue
>    Markus> that ioctl on Linux 2. take the EBS snapshot at any time
>    Markus> 3. clone the EBS snapshot to the new/other VM 4. zfs
>    Markus> import the pool there 5. zfs rollback the filesystem to
>    Markus> the snapshot taken in step 1 (or clone it and use that)
> 
> OK, various tests later: this does not really work. If you create the
> snapshot, and make a backup, the snapshot does not show up on the
> backup (whatever the reason, perhaps the disks so inconsistent zfs has
> to rollback).

I was under the impression that metadata operations are always synchronous in zfs, so since 'zfs snapshot' is such an operation it should be on disk as soon as the command completes. But I've nerver actually confirmed this. So, it could be that zfs snapshots don't get commited to disk immediately after all, or EBS confirmed write/flush commands that we're not commited to whatever is considered stable storage in that cloud. I don't know how well-behaved EBS and its snapshots are when ich comes to flush commands, write order etc. So all just speculation, stopping here...


> But the biggest issue is that if writing is going on and you make the
> EBS snapshot, you can't really mount it. Maybe zfs mounts after hours,
> but I just gave if it didn't mount it after 1.5 hours.

By 'mount' do you mean the import of the pool? Did you use -F on import? In any case, this sounds too long. Was the system doing IO?


> Another interesting thing I've seen was a completely empty drive after
> mount!

That's a bit unspecific. What's empty? Disk full of zeros? Partition full of zeros? Pool without file system on it? Pool with empty filesystems?


> So clearly EBS snapshots on a mounted multi-drive pool don't work.

I think with a lot of writes and transactions, you can't realiably avoid a scenario where zfs can't find a valid or good enough mutual transaction accross all disks. By avoiding writes while doing the EBS snapshots, you could more likely end up with something you can actually import. If you can't avoid the writes, you're out of luck. What you really need in that case is the ability to snapshot all EBS disks as group.

Anyway, with the tools at hand (no IO "freeze" like Linux, no EBS snapshots for groups of disks), I don't think you can acomplish what you originally intended.


Markus