Dump + GZIP

[Roland Smith]

On Thu, Aug 16, 2007 at 09:10:12AM -0400, John Nielsen wrote:
> On Thursday 16 August 2007, Grant Peel wrote:
> > Can I safely pump a filesystem dump through gzip during the dumping
> > process?, or di I need to create the dump first then gzip it after?
> 
> I do it all the time: dump -f - ... | gzip > date_filesystem.dump.gz
> or with bzip2: dump -f - ... | bzip2 > date_filesystem.dump.bz2

Unless you're dumping an unmounted filesystem, add '-L' and '-h 0'. The
first is for dumping a snapshot of a live filesystem, and the second one
is to honor the nodump flag for level 0 dumps.
 
> > Does zipping the dumps cause any headaches at restore time?
> 
> Nope: bzcat date_filesystem.dump.bz2 | restore ... -f -
> 
> > (I currently dump 5 servers worth of data to a raid 5 array, and am about
> > 20% away from running out of disk space).
> >
> > Does gzipping a file give a decent compression ratio?
> 
> Depends on what you're compressing, but generally yes. bzip2 generally 
> compresses better but takes a lot more time, CPU and memory at compression 
> time.

Some time ago I did some tests. Compression with gzip saved 50-60% on
dumps of /, /usr and /var. However, the savings for my /home partition
which contains a lot of digital camera pictures in JPEG format was only
11%.

While bzip2 did 8-10% better (except on /home), it takes a _lot_
longer. So at first I decided to stick with gzip.

Later I decided to skip compression altogether. If the backup medium
becomes corrupted, you might still be able to restore the dump for a
large part. Corruption in a compressed file makes the rest of the file
unreadable. 

If you don't compress the dumps, it's easy to split them in multiple DVD
sized parts. The size of a dump is given in kiB, and it must be a
multiple of the block size, which defaults to 10 kiB. A DVD is 4.7 GB =
(4.7e9/1024//10)*10 = 4589840 kiB. The following command creates
a dump of /home in multiple DVD-sized chunks;

  dump -0 -B 4589840 -C 8 -P 'cat - >home-vol${DUMP_VOLUME}.dump' -h 0 -L /home

You can burn these chunks to DVDs with growisofs, e.g:

  growisofs -dvd-compat -Z /dev/dvd=home-vol0.dump
  growisofs -dvd-compat -Z /dev/dvd=home-vol1.dump
  ...

In this case you do need temporary storage for the dump files. You could
conveivably modify the argument of -P to burn directly with growisofs.

One caution for backups though. Use only programs that are available on
a install CD or in /rescue. Your restore process should not depend on a
port that you need to install first!

Roland
-- 
R.F.Smith                                   http://www.xs4all.nl/~rsmith/
[plain text _non-HTML_ PGP/GnuPG encrypted/signed email much appreciated]
pgp: 1A2B 477F 9970 BA3C 2914  B7CE 1277 EFB0 C321 A725 (KeyID: C321A725)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-questions/attachments/20070816/982e4178/attachment.pgp