Backup and FreeBSD/ZFS

[Matthew Seaman]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 04/02/2010 15:35, Svein Skogen (Listmail Account) wrote:

> On a monthly rotation the tapes are placed in a firetolerant safe. Since
> the most critical thing here is the terabyte (and growing!) of original
> photographs, I'm not thinking about just day-to-day diskfailure or
> pebcaks (proper raid and snapshotting handles that rather well). However
> snapshotting and raid solutions handles the house being on fire rather
> poorly, or should we say "Data integrity and fires, get along like a
> house on fire"? ;)

fire tolerant?  That doesn't sound amazingly effective to me.  Would it
stand up to temperatures in excess of 600degC for more than about 20
minutes?  That's going to be fairly typical in a house fire...

A safe like that is a good idea for local storage of backup media while
it waits to go into the tape library or off-site.  It's a bad idea for
storing your entire archive.

You really want your backups to be stored in an off-site location.
Preferably by a company that has the right sort of secure archive
facilities.  'Off-site' means 'sufficiently far away that any
conceivable disaster can't affect them.'  The gold standard for
'conceivable disasters' is a fully laden and fuelled plane crashing
onto your premises.  I did once have a setup where 'off site' was a
storage company only a couple of streets over, but as their archive was
in a former World War II Bunker some 100 feet underground, that was
acceptable.  Usually you'ld be looking at several miles away at minimum.

> And I think ... everyone on this list can agree that data not properly
> backed up, is a fancy way of saying data not yet lost. ;)
> 
> This is why I'm willing to (and have already) cough up for such
> solutions as autoloaders for my home storage server, however my last
> wrestle with ZFS (on freebsd RELENG_7) left me rather less than
> enthusiastic about the backup options.

Tape libraries are horribly expensive since they're not mass market
items.  They are also intrinsically prone to breaking down or failing
to work quite as well as the salesman implied.  They're the only viable
solution when your storage volumes get really huge, but what is
considered huge nowadays is rather more than terabyte scale.  If you can
get away with just a single tape drive you'll save yourself a lot of
money.

LTO4 tapes are rated at 800--1600GB depending on achievable compression,
so they might be big enough on their own.  As image formats are already
internally compressed, I'd expect them to come in at the low end of
that, which might be tight.  Worth trying out if you can get a drive on
evaluation.

You might want to evaluate getting a bunch of 1TB (or larger) hard dives
- -- either USB or hot-swap SATA.  They don't need to perform particularly
well, but they'd have to be rated for a lot of spin-up/spin-down cycles
(so something aimed at the mobile PC market).

One other thing you should seriously consider is on-line backup.  There
are quite a lot of providers out there, and they should be at least
competitive with running your own dedicated backup system.  They also
generally have the advantage of being instantly available if you need to
recover anything in a hurry.

> Someone told me that Amanda should handle this, and I'm looking into it
> now (especially reading up on what I'd need to do to handle disaster
> recovery), but other options are welcome as well, including the option
> of going Solaris (if someone can point me to proper documentation on how
> to get Solaris to do what I want).

Also checkout Bacula.  I've found Bacula quite a lot easier to manage
than Amanda, especially with tape libraries.

> The box itself is a C2D E7500 with 8GB ram, Asus P5Q Premium (the
> "deluxe" version with fewer NICs is on the BigAdmin HCL, basically an
> intel P45 chipset with sufficient number of pci-express slots, and four
> Marvell Yukon gigabit nics with Marvell Alaska PHY), backed by LSI
> SAS-MPT for the autoloader and SAS-MFI for the disks, and will handle
> SMB/CIFS, NFS, and iSCSI services (and the backups of that data).
> Nothing fancy here, meaning it should hardwarewise be no biggie to get
> it up and running in FreeBSD, Solaris (or leave it on Windows Storage
> server if that's the best solution, even if that means the
> iSCSI-target-service has ... less than stellar performance).
> 
> So, I'm basically looking for pointers on what solutions to consider,
> not looking for a pre-cooked solution. I have sufficient external
> diskspace (still with redundancy) to handle the move-to-new-os-and-fs
> issue...
> 
> Thanks again for taking the time to help me out here. ;)

Hard to know what to advise OS-wise.  FreeBSD will do the job, although
I'm not sure the iSCSI-target stuff is the best available.  So will
Solaris for that matter, although more likely to suffer from hardware
incompatibilites.  I really haven't got a clue about how well Windows
would perform although I personally would avoid it simply because it was
Windows...

	Cheers,

	Matthew

- -- 
Dr Matthew J Seaman MA, D.Phil.                   7 Priory Courtyard
                                                  Flat 3
PGP: http://www.infracaninophile.co.uk/pgpkey     Ramsgate
                                                  Kent, CT11 9PW
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.14 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAktq/HoACgkQ8Mjk52CukIzDHQCfZCFh3+b1r1+rlM4gXg/dKLHi
3EwAn1qyfnPzpvObGUvB0MBMXw3glLdi
=e3iw
-----END PGP SIGNATURE-----