continuous backup solution for FreeBSD

[Jeremy Chadwick]

On Fri, Oct 10, 2008 at 11:29:52AM -0400, Mike Meyer wrote:
> On Fri, 10 Oct 2008 07:41:11 -0700
> Jeremy Chadwick <koitsu at FreeBSD.org> wrote:
> 
> > On Fri, Oct 10, 2008 at 03:53:38PM +0300, Evren Yurtesen wrote:
> > > Mike Meyer wrote:
> > >> On Fri, 10 Oct 2008 02:34:28 +0300
> > >> yurtesen at ispro.net wrote:
> > >>
> > >>> Quoting "Oliver Fromme" <olli at lurza.secnetix.de>:
> > >>>
> > >>>> These features are readily available right now on FreeBSD.
> > >>>> You don't have to code anything.
> > >>> Well with 2 downsides,
> > >>
> > >> Once you actually try and implement these solutions, you'll see that
> > >> your "downsides" are largely figments of your imagination.
> > >
> > > So if it is my imagination, how can I actually convert UFS to ZFS  
> > > easily? Everybody seems to say that this is easy and that is easy.
> > 
> > It's not that easy.  I really don't know why people are telling you it
> > is.
> 
> Maybe because it is? Of course, it *does* require a little prior
> planning, but anyone with more than a few months experience as a
> sysadmin should be able to deal with it without to much hassle.
> 
> > Converting some filesystems are easier than others; /home (if you
> > create one) for example is generally easy:
> > 
> > 1) ZFS fs is called foo/home, mounted as /mnt
> > 2) fstat, ensure nothing is using /home -- if something is, shut it
> >    down or kill it
> > 3) rsync or cpdup /home files to /mnt
> > 4) umount /home
> > 5) zfs set mountpoint=/home foo/home
> > 6) Restart said processes or daemons
> > 
> > "See! It's like I said! EASY!"  You can do this with /var as well.
> 
> Yup. Of course, if you've done it that way, you're not thinking ahead,
> because:
> 
> > Now try /usr.  Hope you've got /rescue available, because once /usr/lib
> > and /usr/libexec disappear, you're in trouble.  Good luck doing this in
> > multi-user, too.
> 
> Oops. You F'ed up. If you'd done a little planning, you would have
> realized that / and /usr would be a bit of extra trouble, and planned
> accordingly.
> 
> > And finally, the root fs.  Whoever says "this is easy" is kidding
> > themselves; it's a pain.
> 
> Um, no, it wasn't. Of course, I've been doing this long enough to have
> a system set up to make this kind of thing easy. My system disk is on
> a mirror, and I do system upgrades by breaking the mirror and
> upgrading one disk, making everything work, then putting the mirror
> back together. And moving to zfs on root is a lot like a system
> upgrade:
> 
> 1) Break the mirror (mirrors actually, as I mirrored file systems).
> 2) Repartition the unused drive into /boot, swap & data
> 3) Build zfs & /boot according to the instructions on ZFSOnRoot
>    wiki, just copying /boot and / at this point.
> 4) Boot the zfs disk in single user mode.
> 5) If 4 fails, boot back to the ufs disk so you're operational while
>    you contemplate what went wrong, then repeat step 3. Otherwise, go
>    on to step 6.
> 6) Create zfs file systems as appropriate (given that zfs file
>    systems are cheap, and have lots of cool features that ufs
>    file systems don't have, you probably want to create more than
>    you had before, doing thing like putting SQL serves on their
>    own file system with appropriate blocking, etc, but you'll want to
>    have figured all this out before starting step 1).
> 7) Copy data from the ufs file systems to their new homes,
>    not forgetting to take them out of /etc/fstab.
> 8) Reboot on the zfs disk.
> 9) Test until you're happy that everything is working properly,
>    and be prepared to reboot on the ufs disk if something is broken. 
> 10) Reformat the ufs disk to match the zfs one. Gmirror /boot,
>     add the data partition to the zfs pool so it's mirrored, and
>     you should have already been using swap.
> 
> This is 10 steps to your "easy" 6, but two of the extra steps are
> testing you didn't include, and 1 of the steps is a failure recovery
> step that shouldn't be necessary. So - one more step than your easy
> process.

Of course, the part you seem to be (intentionally?) forgetting: most
people are not using gmirror.  There is no 2nd disk.  They have one disk
with a series of UFS2 filesystems, and they want to upgrade.  That's how
I read Evren's "how do I do this? You say it's easy..." comment, and I
think his viewpoint is very reasonable.

> Yeah, this isn't something you do on a whim. On the other hand, it's
> not something that any competent sysadmin would consider a pain. For a
> good senior admin, it's a lot easier than doing an OS upgrade from
> source, which should be the next step up from trivial.

I guess you have a very different definition of "easy".  :-)

The above procedure, in no way shape or form, will be classified as
"easy" by the user (or even junior sysadmin) community, I can assure you
of that.

I'll also throw this in the mix: the fact that we are *expecting* users
to know how to do this is unreasonable.  It's even *more* rude to expect
that mid-level or senior SAs have to do it "the hard way".  Why?  I'll
explain:

I'm an SA of 16+ years.  I'm quite familiar with PBR/MBR, general disk
partitioning, sectors vs. blocks, slices, filesystems, and whatever
else.  You want me to do it by hand, say, with bsdlabel -e?  Fine, I
will -- but I will not be happy about it.  I have the knowledge, I
know how to do it, so why must the process continue to be a PITA and
waste my time?

This is exactly why I hate things like the OpenBSD installer: do not
make me do it the hard way.  We're living in 2008, not 1989.  Evolve.

-- 
| Jeremy Chadwick                                jdc at parodius.com |
| Parodius Networking                       http://www.parodius.com/ |
| UNIX Systems Administrator                  Mountain View, CA, USA |
| Making life hard for others since 1977.              PGP: 4BD6C0CB |