continuous backup solution for FreeBSD

[Matthew Dillon]

    Go here:

	http://forums.mysql.com/read.php?28,197644,197644

    There are a ton of ways to maintain mysql backups without having to
    replay the entire log.  Google some keywords.

    With regards to solutions based on filesystem snapshots, such as partial
    log replaying (snapshot + new-log, then replay from snapshot after crash),
    HAMMER and ZFS are your only real choices in the BSD world.  UFS
    snapshots have all sorts of issues that make them unsuitable.  Block level
    snapshots are unreliable.

    When interacting with a high level program for crash recovery purposes
    you really want to use a filesystem's native snapshot capability
    (e.g. HAMMER or ZFS.  UFS's isn't good enough).  You do not need to
    explicitly backup the filesystem to other media.  That is, you of course
    still make such backups, but they would only be used in case of a
    catastrophic loss of the production filesystem, not for simple crash
    recovery which can be done from the same-media snapshots.

					--

    In anycase, HAMMER has two native backup solutions.  Both are easy to
    use.

    (1) Just use cpdup.  This works nicely as the source filesystem, if
	HAMMER, can be snapshotted, and then used as a stable cpdup source.
	cpdup to the target and let HAMMER manage the history for you.  No
	hardlink trick needed or anything like that.  Use HAMMER's
	snapshots on the target to determine what history you want to
	retain.

	Advantages: Extremely convenient, target storage is reasonably
	efficient.  Great if sources are mixed filesystem types.

	Disadvantages: Runs cpdup so you are scanning the directory hierarchy
	on both.  Inode numbers not retained, so not suitable for fallback.

	(this sort of methodology can be used w/ ZFS too).

    (2) For HAMMER-to-HAMMER you can use HAMMER's native mirroring.  Simply
	create a PFS (pseudo hammer filesystem) slave on the target, which
	takes 2 seconds, then use the 'hammer mirror-copy' or
	'hammer mirror-stream' directives (which can take remote host
	specifications and will run ssh) to mirror from source to target.
	HAMMER's mirroring is incremental from the protocol right down
	to the media accesses.  No pre-scan occurs is needed.

	Advantages: Provides bandwidth-controlled incremental streaming,
	near-realtime operation (30-60 second lag though finer-grained
	operation is possible).  Extremely robust (can be ^C'd and restarted,
	or left offline for long periods of time, etc).  Mirrors all
	fine-grained history on source and can be re-pruned on the target
	to the desired interval.  Has little effect on production machines
	(no queues means backups can't feed-back and effect the performance
	of the production box).  Mirrors inode numbers etc.  Mirroring
	target can become a new master in a pinch (but can't be made a slave
	again, sorry).  Should serve the same NFS fsid, etc.

	Disadvantages: Must mirror the entire PFS, only works between HAMMER
	filesystems.  Near-real-time (30-60 second lag) is not real time, but
	it's probably close enough.

	(ZFS has a way to do something similar but I do not know what the
	various advantages or disadvantages of using the feature are).

    DragonFly also has real-time journaling at the VFS layer, which is not
    HAMMER-specific, but it requires queuing and isn't really suitable on
    a production environment for off-machine copying because the queue
    can fill up and block the filesystem.  HAMMER's mirroring is far more
    robust.

						-Matt