zfs q regarding backup strategy

[David Christensen]

On 10/1/21 07:14, tech-lists wrote:
> Hello,
> 
> On Thu, Sep 30, 2021 at 08:06:28PM -0700, David Christensen wrote:
> 
>> Michael Warren Lucas is my go-to author for FreeBSD and ZFS:
> 
> I think those were written quite a while ago? I need something covering
> both the old zfs and OpenZFS. I'm looking for the differences, any
> incompatibilities that might apply in my context. It'd be great if he
> has more recent stuff.


I find MWL's writings to be recent enough.  He updates his books.  I 
expect he has a process for updating blog posts.


>> I use zfs-auto-snapshot for snapshot creation/ management, and homebrew
>> commands/ scripts for replication.  The homebrew stuff is problematic.
> 
> yeah homebrew I'm trying to avoid for lots of reasons.
> 
>> I need to evaluate something like zfs-replicate (available as a FreeBSD
>> package of the same name):
> 
> Thanks for the suggestion. Problem with zfs-replicate is its dependence
> on python, which from what I've observed historically, is a bit of a mess.
> On my desktop there's lots of ports still needed that are dependent on
> python2 even though upstream have retired it. They show up in
> reports as EoL'd due to python2; there's loads right now and the desktop
> builds its ports in poudriere weekly.


Thankfully, I can find what I need in packages; and zfs-replicate is a 
current package.


> What I'm looking for is something (ideally) written in sh. Or even perl.


Available in the links I already posted.


> But even before that, need to understand incremental backups properly.


I learned by beating my head against the console.  :-/


> Basically the backup needs to be (effectively) write-only. By that, I
> mean only added to.


ZFS snapshots are read-only by design.


When you replicate a read-write filesystem, you replicate via snapshots. 
   The snapshots arrive read-only, and the filesystem arrives read-write.


If you touch the destination filesystem, the next incremental 
replication will fail.  The '-F' option to 'zfs receive' is required to 
discard changes on the destination.


I do not see an option for 'zfs receive' to set properties during 
replication, but you could write a script to set 'readonly=on' on the 
destination after replication.  If so, I don't know what happens on the 
next incremental replication -- properties are overwritten?  Replication 
fails?  Testing is required.


Eventually, the destination will fill up.  Choices include replacing the 
media and deleting snapshots.  I do the latter (by hand).


> Maybe zfs snapshots aren't the way to go; 


Snapshots and replication are killer features of ZFS, and primary 
reasons why I use ZFS.


> maybe rsync in the backup
> direction only (apart from when something needs to be restored) would
> suit my context better. 


I use rsync(1) to backup from non-ZFS sources to ZFS.


> What do you think?


"More than enough rope to shoot yourself in the foot".


David