zfs q regarding backup strategy

[David Christensen]

On 9/30/21 16:48, tech-lists wrote:
> Hi,
> 
> This is a bit chicken-and-egg for me, in that I might be formulating the
> wrong question, and so getting wrong answers, so I apologise in advance 
> if I'm being dense.
> 
> Basically the end result I want is to be in a position at a backup site
> with full backups that can be restored to the live site if needed, or 
> even to get just one required file from that backup, given the filename 
> and modification time. The backup site has very much more diskspace than
> the live site.
> 
> I know how to make the full backup and to send it. What I'm looking for
> is incremental (or would it be differential) zfs backups (snapshots). So 
> let's say an incremental/differential backup to the remote site happens 
> every 6hrs.
> 
> If 10 incremential backups happen after the full one, and on the
> live service someone loses a few files, later adding another few, maybe
> a couple of those have the same name as before just with a different
> modification time, all the while this is happening 24hrs pass.
> 
> How are the accumulated differential/incremental snaps incorporated into
> the main backup? Would I need to run something else on the backup
> server to integrate the snaps with the full backup? Or would all the 
> snaps need to be kept?
> How would one tell if the required files in this example are in one snap
> and not in the others?
> 
> If the policy was to keep everything in backup regardless of whether it
> was deleted or not from the live service (so the backup is like a 
> write-only representation) I guess the approach would be different to 
> say if the backup just had to be the latest representation of the data 
> on the live server. But if it's "keep everything" then what happens
> if the live server makes foo.txt then deletes it then makes foo.txt 10
> days later in the same place? All the while these
> (incremental/differential) snaps have been arriving every 6 hrs.
> 
> Additionally the live site is the old zfs on releng/12.2-p10 and the 
> backup site is with the OpenZFS on stable/13 which might be an issue
> if they handle things significantly differently.
> 
> thanks advance for any suggestions or any scripts/tools you can
> recommend.


I have a SOHO environment with a FreeBSD 12.2-RELEASE machine providing 
Samba and CVS services.


Michael Warren Lucas is my go-to author for FreeBSD and ZFS:

https://mwl.io/nonfiction/os

https://mwl.io/archives/2128

https://mwl.io/archives/2140


I use zfs-auto-snapshot for snapshot creation/ management, and homebrew 
commands/ scripts for replication.  The homebrew stuff is problematic. 
I need to evaluate something like zfs-replicate (available as a FreeBSD 
package of the same name):

https://pypi.org/project/zfs-replicate/


David