Make ZFS auto-destroy snapshots when the out of space?

[Xin LI]

Hi, Kirk,

On Sat, May 29, 2010 at 1:07 PM, Kirk Strauser <kirk at strauser.com> wrote:
> I found some nice scripts to regularly snapshot all the filesystems in my
> ZFS pool at
> http://www.neces.com/blog/technology/integrating-freebsd-zfs-and-periodic-snapshots-and-scrubs
> . One thing bothers me, though: I have to intentionally set how many months'
> worth of snapshots I want to keep. Too many and I run out of room. Too few
> and I lose some of the benefits of easy recovery of deleted data. My
> computer is better at bookkeeping than I am, so why not let it?
>
> I'd propose standardizing on an attribute like org.freebsd:allowautodestroy.
> Modify ZFS's disk full behavior to scan for snapshots with that attribute
> set and destroy the oldest one, and continue until there's enough free space
> to complete a write requests or until out of "expendable" snapshots to
> destroy (at which time the normal disk full handler would run). Also run a
> daily periodic script to ensure that the free space stays below a
> configurable threshold each day so that ZFS isn't constantly butting up
> against completely full drives.
>
> This would take all configuration guesswork out of the equation and would
> let me keep as many snapshots as I have space to maintain. If I want to
> extend my reach back in time, I can add another drive to the pool and the
> rest is handled automatically. At the same time, should I suddenly *want* to
> store massive amounts of new data, the snapshots can be easily and
> automatically cleared out to make room for the stuff I want to hold.
>
> What do you think? It seems like this should be pretty easy to implement
> without requiring any upstream changes or new FreeBSD-only data structures.
> The whole thing could possibly be implemented in userspace, but I don't know
> that ZFS has any exception handling callbacks that would make it easy.
>
> An unused resource is a wasted resource, right?

I think this sounds like a good idea but I think we may probably want
to explore a more general mechanism, e.g. a daemon that "listen"s
system events like file system full, etc. and execute some user
defined actions.

One thing that we want to avoid is that by making the "automatic
recycle" we would open a new race between system and user backup
programs, i.e. if you remove an intermediate snapshot, 'zfs send' may
fail at receiving side, if incremental send is being used.  We would
need a way to "notify" that a 'zfs send' is underway.

Cheers,
-- 
Xin LI <delphij at delphij.net> http://www.delphij.net