Anyone used rsync scriptology for incremental backup?

[Freddie Cash]

On Thu, Oct 30, 2008 at 1:50 AM, Andrew Snow <andrew at modulus.org> wrote:
> In this way, each day we generate a batch file that lets us step back one
> day.  The diffs themselves, compressed with gzip, and extremely space
> efficient.  We can step back potentially hundreds of days, though it seems
> to throw errors sometimes when backing up Windows boxes, which I haven't
> tracked down yet.
>
> But to be honest, soon you can save yourself a lot of hassle by simply using
> ZFS and taking snapshots.  It'll be faster, and with compression very space
> efficient.

That's exactly what we do, use ZFS and RSync.  We have a ZFS
/storage/backup filesystem, with directories for each remote site, and
sub-directories for each server to be backed up.

Each night we snapshot the directory, then run rsync to backup each
server.  Snapshots are named with the current date.  For 80 FreeBSD
and Linux servers, we average 10 GB of changed data a night.

No muss, no fuss.  We've used it to restore entire servers (boot off
Knoppix/Frenzy CD, format partitions, rsync back), individual files
(no mounting required, just cd into the .zfs/snapshot/snapshotname
directory and scp the file), and even once to restore the permissions
on a pair of servers where a clueless admin "chmod -R user /home" and
"chmod -R 777 /home".

Our backup script is pretty much just a double-for loop that scans a
set of site-name directories for server config files, and runs rsync
in parallel (1 per remote site).

We we looking into using variations on rsnapshot, custom
squashfs/hardlink stuff, and other solutions, but once we started
using ZFS, we stopped looking down those roads.  We were able to do in
3 days of testing and scripting what we hadn't been able to do in
almost a month of research and testing.

-- 
Freddie Cash
fjwcash at gmail.com