Musings on ZFS Backup strategies

[Peter Jeremy]

On 2013-Mar-01 08:24:53 -0600, Karl Denninger <karl at denninger.net> wrote:
>If I then restore the base and snapshot, I get back to where I was when
>the latest snapshot was taken.  I don't need to keep the incremental
>snapshot for longer than it takes to zfs send it, so I can do:
>
>zfs snapshot pool/some-filesystem at unique-label
>zfs send -i pool/some-filesystem at base pool/some-filesystem at unique-label
>zfs destroy pool/some-filesystem at unique-label
>
>and that seems to work (and restore) just fine.

This gives you an incremental since the base snapshot - which will
probably grow in size over time.  If you are storing the ZFS send
streams on (eg) tape, rather than receiving them, you probably still
want the "Towers of Hanoi" style backup hierarchy to control your
backup volume.  It's also worth noting that whilst the stream will
contain the compression attributes of the filesystem(s) in it, the
actual data is the stream in uncompressed

>This in turn means that keeping more than two incremental dumps offline
>has little or no value; the second merely being taken to insure that
>there is always at least one that has been written to completion without
>error to apply on top of the base.

This is quite a critical point with this style of backup: The ZFS send
stream is not intended as an archive format.  It includes error
detection but no error correction and any error in a stream renders
the whole stream unusable (you can't retrieve only part of a stream).
If you go this way, you probably want to wrap the stream in a FEC
container (eg based on ports/comms/libfec) and/or keep multiple copies.

The "recommended" approach is to do zfs send | zfs recv and store a
replica of your pool (with whatever level of RAID that meets your
needs).  This way, you immediately detect an error in the send stream
and can repeat the send.  You then use scrub to verify (and recover)
the replica.

>(Yes, I know, I've been a ZFS resister.... ;-))

"Resistance is futile." :-)

On 2013-Mar-01 15:34:39 -0500, Daniel Eischen <deischen at freebsd.org> wrote:
>It wasn't clear that snapshots were traversable as a normal
>directory structure.  I was thinking it was just a blob
>that you had to roll back to in order to get anything out
>of it.

Snapshots appear in a .zfs/snapshot/SNAPSHOT_NAME directory at each
mountpoint and are accessible as a normal read-only directory
hierarchy below there.  OTOH, the send stream _is_ a blob.

>Am I correct in assuming that one could:
>
>   # zfs send -R snapshot | dd obs=10240 of=/dev/rst0
>
>to archive it to tape instead of another [system:]drive?

Yes.  The output from zfs send is a stream of bytes that you can treat
as you would any other stream of bytes.  But this approach isn't
recommended.

-- 
Peter Jeremy
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 196 bytes
Desc: not available
URL: <http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20130303/1462fdb5/attachment.sig>