Trials and tribulations on mirroring....

[Jon Noack]

On 01/18/05 23:46, Karl Denninger wrote:
> <snip backstory and gvinum stuff>
> 
> So, my next attempt (with a new, fresh load) was to give gmirror a shot
> instead, since I only need Raid 1, and this looks like a "better" option
> from that perspective given gvinum's squirrely and not-all-there status at
> the moment.
> 
> Weeeeellll.. the manual page (and the archives on the list here) implies that 
> I can do the following:
> 
> 1. Set up the system on one of the two mirror disks.
> 
> 2. In single user mode, use "gmirror label -v -b round-robin disk ad4"
>    to set up the pre-existing disk (with data on it), which I booted from,
>    as the "primary".
> 
> 3. Then use "gmirror insert disk ad6" to add the secondary (mirrored) disk,
>    which will automatically sync it and bring them into a consistent state.
> 
> Only one small problem - the first command (in #2) fails, with:
> 
> "Can't store metadata on ad4: Operation not permitted"
> 
> This is true if the system is mounted read-only (in single user mode) or
> read-write (in multiuser, with all filesystems mounted)
> 
> Attempting to label the SECOND (bare) disk works, but of course that's 
> backwards and an "insert" would immediately destroy the running system (I
> assume it would also fail with the same permission error, but I've not tried
> it) - so that's not an option.
> 
> I also tried it from the fixit disk, but the KLD cannot be loaded from the
> fixit disk, and none of the commands work - so that appears to not be an 
> option either.
> 
> How do you get this thing set up?  The archives here (and man pages) 
> strongly imply that this works for a root filesystem and boot drive - 
> what am I missing?
> 
> Need a Raid 1 solution that works to move over to the 5.x world out here...

If any filesystem is mounted then GEOM will not allow metadata updates 
for the drive.  Here's a solution given over on the geom@ list:
http://lists.freebsd.org/pipermail/freebsd-geom/2004-November/000379.html

I converted to gmirror from ataraid, so I already had 2 disks with 
identical contents.  I worked around this problem as outlined here (the 
instructions have errors that are corrected later in the thread):
http://lists.freebsd.org/pipermail/freebsd-current/2004-October/039907.html

I wish we handled this in the installer, but as I can't produce the code 
to do it I'll shut up now...

Jon