lazy mirror / live backup

[Eric Anderson]

On 04/20/07 20:25, Mike Wolman wrote:
> 
> On Sat, 21 Apr 2007, Bernd Walter wrote:
> 
>> On Sat, Apr 21, 2007 at 01:38:31AM +0100, Mike Wolman wrote:
>>> My only concern is zfs is quite heavy weight (memory wise) compared to
>>> gmirror and ufs for a simple laptop/desktop setup which you may just want
>>> to replicate the entire drive and grab the ggatec device and be ready to
>>> run should anything happen to the machine.
>> Yes - that's unfortunately true.
>>
>>> Im sure there are other situations where running zfs on the available
>>> hardware is not an option compared to gmirror - im not sure what the
>>> recommended amount for freebsd but as far as i can rember the suggested
>>> about for solaris is 1Gb - comparing to gmirror i think i have run it on a
>>> machine doing simple home fileserving with 128Mb.
>> Ever thought about UFS snapshots backed with rsync?
>> You get a consitent pseudo image of your running filesystem with
>> unallocated blocks represented as zeros.
>> rsync now allows comparing file chunks and copies only differences.
>> Still every block need to be read, though.
> 
> Yea i use rsync and snapshots quite a bit, but unfortunately rsync works 
> at the filesystem level so you cant really get a bootable image of the whole 
> device.  It would be nice if this could be done without user interaction, 
> ie if the ggatec component of a mirror disappears and reappears gmirror 
> justs gets to work syncing things up.

rsync and it's like are great tools, but traversing the tree every time 
is not only slow and heavy handed, but it takes an enormous amount of 
memory (akin to fsck) for large file systems.  rsync isn't the right 
tool for lazy syncing.

>> vbackup from devel/plan9port stores checksums and allows offering
>> only different blocks to the other side.
>> venti - the backing store behind vbackup - allows compression and
>> single storage of different blocks with same data, which reduces
>> the required backup capacity very impressive.
>> My expirience with vbackup is that this mechanism is fast enough
>> as long as there are no hughe differences.
>>
> 
> I'll take a peek as it sounds interesting,

Eric