Backup advice

[Doug Hardie]

On May 23, 2007, at 16:27, Jason Lixfeld wrote:

> So I feel a need to start backing up my servers.  To that end, I've  
> decided that it's easier for me to grab an external USB drive  
> instead of a tape.  It would seem dump/restore are the tools of  
> choice.  My backup strategy is pretty much "I don't want to be  
> screwed if my RAID goes away".  That said I have a few questions  
> along those lines:
>
> - Most articles I've read suggest a full backup, followed by  
> incremental backups.  Is there any real reason to adopt that format  
> for a backup strategy like mine, or is it reasonable to just do a  
> dump 0 nightly?  I think the only reason to do just one full backup  
> per 'cycle' would be to preserve system resources, as I'm sure it's  
> fairly taxing on the system during dump 0 times.
>
> - Can dump incrementally update an existing dump, or is the idea  
> that a dump is a closed file and nothing except restore should ever  
> touch it?
>
> - How much does running a backup through gzip actually save?  Is  
> taxing the system to compress the dump and the extra time it takes  
> actually worth it, assuming I have enough space on my backup drive  
> to support a dump 0 or two?
>
> - Other folks dumping to a hard drive at night?  Care to share any  
> of your experiences/rationale?

The criteria for selecting a backup approach is not the backup  
methodology but the restore methodology.  What failures can you  
tollerate and what can you not afford to lose forever.  Backup to a  
single disk leaves you with a big vulnerability if something is wrong  
with that backup.  You stand to lose pretty much everything.  If  
everything is stored in one location, what happens if it vanishes?

My approach is dictated by the restore requirements.  We have some  
databases that are absolutely critical.  Loss of those is the end of  
the world.  Every module that updates the database also writes a copy  
of the updated transaction to a log file.  I rsync the log file to  
multiple machines separated by many miles every 10 minutes.  The  
complete database is dumped everynight and that is also rsync'd to  
the same machines daily.  Each of the backup machines retains several  
months of the full dumps and the transaction logs.  From that  
presuming that one site remains available, I can reconstruct all but  
the last 10 minutes of the database.

The complete system is dumped to a disk on one of the local servers  
weekly.  A DVD is cut from that and taken off-site for retention.   
Actually all that is needed is the local software source and the  
config files as FreeBSD is easily replaced.  However, since there are  
always times were some of the ports may not be the latest version it  
is easier to have the actual ones in use rather than having to  
checkout newer versions.  The full dump is also rsync'd weekly to a  
couple of off-site machines.

Whatever you decide to do, figure out how to recover and test it.   
Finding out you need something you didn't save is much less traumatic  
if you find out before a failure occurs.  I test my restore  
procedures yearly.  I have two machines I use at home for doing test  
recoveries.