ZFS replication suggestions

[N.J. Thomas]

* mike tancsa <mike at sentex.net> [2019-06-21 15:25:20-0400]:
> was wondering what people are using these days to sync their ZFS files
> systems between servers?

I use a combination of syncoid (the replication subsystem of Sanoid) and
zxfer.

Syncoid doesn't delete snapshots it has not created itself, so because I
use zfstools to create my snapshots, syncoid won't delete these --- I
don't consider this a bad thing necessarily, the author is very
cautious.

So what I do is use syncoid to replicate my snapshots to a backup server
(called A), and use zxfer to prune older snapshots. The zxfer runs right
after syncoid, so it really only just deleting old snaps. (NB: This is
technically not a backup on server A, it's just a sync.).

I use syncoid to replicate everything on backup server A to two other
backup servers (B and C, with server C being offsite), ie. backups of
the backup server. There is no automatic deletion of older snaps on
these servers. So this is my real backup.

Every couple of months, I'll go in and manually prune older stuff on B
and C. I'm told that once ZFS gets above 10,000 snapshots, bad things
happen, so I try not to go above 10k snaps on any one partition.
Manually pruning by hand about every 2 months seems to work fine.

The reason I don't just use zxfer for everything: syncoid uses mbuffer
and lzop, so it's very fast, it can also resume partially replicated
transfers and is aware of zfs bookmarks, etc.

One last thing to note, zfstools and zxfer is in ports. Sanoid is not. I
clone sanoid from Github, and use a small patch to fix the shebang line
and some minor things in the syncoid script to make it work on BSD (it's
written in Perl and assumes Linux).

Thomas