Backup and FreeBSD/ZFS

Thu Feb 4 18:14:20 UTC 2010

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 04.02.2010 17:57, Matthew Seaman wrote:
> On 04/02/2010 15:35, Svein Skogen (Listmail Account) wrote:
> 
>> On a monthly rotation the tapes are placed in a firetolerant safe. Since
>> the most critical thing here is the terabyte (and growing!) of original
>> photographs, I'm not thinking about just day-to-day diskfailure or
>> pebcaks (proper raid and snapshotting handles that rather well). However
>> snapshotting and raid solutions handles the house being on fire rather
>> poorly, or should we say "Data integrity and fires, get along like a
>> house on fire"? ;)
> 
> fire tolerant?  That doesn't sound amazingly effective to me.  Would it
> stand up to temperatures in excess of 600degC for more than about 20
> minutes?  That's going to be fairly typical in a house fire...

Well, this one is the kind placed within the concrete of our cellar
(this is a home solution, not an industry one). But that area isn't
suitable for the servers for other reasons. The cellar is within the
bedrock of the area (the house foundation is directly on bedrock, and
the cellar area has been blasted out from the bedrock), so discounting
the plane-crash-into-building scenario, it's rather safe for our use
(and the plane-crash scenario would quite likely invalidate me along
with the backup, and so the need for a restore wouldn't be that critical)

> A safe like that is a good idea for local storage of backup media while
> it waits to go into the tape library or off-site.  It's a bad idea for
> storing your entire archive.
> 

*snip*

> 
> Tape libraries are horribly expensive since they're not mass market
> items.  They are also intrinsically prone to breaking down or failing
> to work quite as well as the salesman implied.  They're the only viable
> solution when your storage volumes get really huge, but what is
> considered huge nowadays is rather more than terabyte scale.  If you can
> get away with just a single tape drive you'll save yourself a lot of
> money.

Alas, a full backup of the current disk setup takes 4 tapes and ... I
really don't feel like staying up one entire night per week to swap
tapes (both for the backup and the verify). The autoloader I've got now
(8 slot, 1 drive, LTO-3, SAS) works fairly well with the currently
installed OS (Windows Storage Server 2008), giving about 60MB/Sec
sustained transfer rate.

> LTO4 tapes are rated at 800--1600GB depending on achievable compression,
> so they might be big enough on their own.  As image formats are already
> internally compressed, I'd expect them to come in at the low end of
> that, which might be tight.  Worth trying out if you can get a drive on
> evaluation.

A standalone LTO-4 might be a good alternative, if I didn't already have
the tapeloader. ;)

> You might want to evaluate getting a bunch of 1TB (or larger) hard dives
> -- either USB or hot-swap SATA.  They don't need to perform particularly
> well, but they'd have to be rated for a lot of spin-up/spin-down cycles
> (so something aimed at the mobile PC market).
> 
> One other thing you should seriously consider is on-line backup.  There
> are quite a lot of providers out there, and they should be at least
> competitive with running your own dedicated backup system.  They also
> generally have the advantage of being instantly available if you need to
> recover anything in a hurry.

Online-backup-solutions are a no-go for me, alas.

>> Someone told me that Amanda should handle this, and I'm looking into it
>> now (especially reading up on what I'd need to do to handle disaster
>> recovery), but other options are welcome as well, including the option
>> of going Solaris (if someone can point me to proper documentation on how
>> to get Solaris to do what I want).
> 
> Also checkout Bacula.  I've found Bacula quite a lot easier to manage
> than Amanda, especially with tape libraries.
> 
>> The box itself is a C2D E7500 with 8GB ram, Asus P5Q Premium (the
>> "deluxe" version with fewer NICs is on the BigAdmin HCL, basically an
>> intel P45 chipset with sufficient number of pci-express slots, and four
>> Marvell Yukon gigabit nics with Marvell Alaska PHY), backed by LSI
>> SAS-MPT for the autoloader and SAS-MFI for the disks, and will handle
>> SMB/CIFS, NFS, and iSCSI services (and the backups of that data).
>> Nothing fancy here, meaning it should hardwarewise be no biggie to get
>> it up and running in FreeBSD, Solaris (or leave it on Windows Storage
>> server if that's the best solution, even if that means the
>> iSCSI-target-service has ... less than stellar performance).
> 
>> So, I'm basically looking for pointers on what solutions to consider,
>> not looking for a pre-cooked solution. I have sufficient external
>> diskspace (still with redundancy) to handle the move-to-new-os-and-fs
>> issue...
> 
>> Thanks again for taking the time to help me out here. ;)
> 
> Hard to know what to advise OS-wise.  FreeBSD will do the job, although
> I'm not sure the iSCSI-target stuff is the best available.  So will
> Solaris for that matter, although more likely to suffer from hardware
> incompatibilites.  I really haven't got a clue about how well Windows
> would perform although I personally would avoid it simply because it was
> Windows...

Windows was chosen as ... the least painful alternative at the time, and
luckily I'm pragmatic about computing OS'es. They're all broken and
prone to crashes. :p

//Svein

- -- 
- --------+-------------------+-------------------------------
  /"\   |Svein Skogen       | svein at d80.iso100.no
  \ /   |Solberg Østli 9    | PGP Key:  0xE5E76831
   X    |2020 Skedsmokorset | svein at jernhuset.no
  / \   |Norway             | PGP Key:  0xCE96CE13
        |                   | svein at stillbilde.net
 ascii  |                   | PGP Key:  0x58CD33B6
 ribbon |System Admin       | svein-listmail at stillbilde.net
Campaign|stillbilde.net     | PGP Key:  0x22D494A4
        +-------------------+-------------------------------
        |msn messenger:     | Mobile Phone: +47 907 03 575
        |svein at jernhuset.no | RIPE handle:    SS16503-RIPE
- --------+-------------------+-------------------------------
         If you really are in a hurry, mail me at
               svein-mobile at stillbilde.net
 This mailbox goes directly to my cellphone and is checked
        even when I'm not in front of my computer.
- ------------------------------------------------------------
                     Picture Gallery:
          https://gallery.stillbilde.net/v/svein/
- ------------------------------------------------------------
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.12 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAktrDnkACgkQODUnwSLUlKR8swCgs20KjWiGKoNJK/llELC3PcNL
CyoAoLkDFCvYU8NyI80gF5RVxuo5FWcH
=X40n
-----END PGP SIGNATURE-----