gmirror oddities

[George Hartzell]

Eirik Øverby writes:
 > Hi!
 > 
 > I've been using gmirror for a while to safeguard my system disks. I have
 > taken the slice-based mirror approach, where I use, say, ad0s1 and ad2s1 as
 > providers.
 > On one of my servers, this seems to be impossible. I create the mirror using
 > ad2s1 first (to keep my system running while I do some of the work), and
 > then I re-initialize ad0s1 (making it exactly the size of ad2s1) before
 > using gmirror insert to add it to the mirror.
 > However, at this point - when doing a gmirror list - it turns out that it
 > never added ad0s1 as a provider, but ad0 itself! As a result, I now have a
 > load of slices (ad0a, ad0b, ad0d, ad0e, ad0f) instead of having the same
 > structure as I have on ad2s1. It's just like ad2s1, just without the "s1"
 > part.
 > 
 > I've tried "dd if=/dev/zero of=/dev/ad0 bs=65536" a couple of times, in case
 > some old provider metadata was stored there. I also have exactly the same
 > setup in another server, the only difference being that it behaves as
 > expected..
 > 
 > Am I doing something blatantly wrong here? This IS supposed to work, right?
 > I've even found a very nice description of how to do it at
 > http://people.freebsd.org/~rse/mirror/
 > confirming that what I'm doing is right.
 > 
 > I'm on 5.4-PRERELEASE, but this problem has been there since 5.3-p2 or
 > something, which was when I first tried this.

I bet you're getting bitten by a problem that bit me.  It's described
in the fine print in http://people.freebsd.org/~rse/mirror/.

Gmirror saves it's metadata on the last sector of its disk space.
Since the slice (adXs1) and the disk device (adX) end at the same
place on the disk, gmirror gets confused.  It tastes devices in a
particular order, apparently devices first, then slices.  It finds the
metadata when it tastes adX and goes ahead and uses it, even though it
should be associating it w/ adXsY.  Hilarity ensues....

The fix is described in the fourth comment block of Ralf's doc, either
make the slice a sector smaller than the disk device or hardcode the
provider name.  I've been using the hardcoding approach, and it seems
to work for me.

g.