Gmirror question - what am I doing wrong?

[Karl Denninger]

I attempted to recreate this failure, and was unable to do so.  Not quite
sure why, but in several trials over the last 48 hours I was able to remove
a disk from the running system, replace it with either the large or small
capacity replacement, re-attach it and have it properly resync.

I'm not quite ready to say 'all is ok', but am fairly close... :)

Sorry 'bout the (apparently false) alarm......

--
-- 
Karl Denninger (karl at denninger.net) Internet Consultant & Kids Rights Activist
http://www.denninger.net	My home on the net - links to everything I do!
http://scubaforum.org		Your UNCENSORED place to talk about DIVING!
http://www.spamcuda.net		SPAM FREE mailboxes - FREE FOR A LIMITED TIME!
http://genesis3.blogspot.com	Musings Of A Sentient Mind

On Thu, Jan 27, 2005 at 12:05:29AM -0600, Karl Denninger wrote:
> Ok, so I have Gmirror working for two exactly-equal sized disks, either on
> the raw device or on the slice.
> 
> Here's the issue.
> 
> I have two vendors of a particuar size disk.  They have slightly different
> actual sizes, and detect as slightly different in sysinstall and FDISK.
> The displayed geometries differ only in cylinder counts, BUT the partition
> function in either sysinstall or FDISK (whether I use the command line or
> the "tool") will not permit an EXACT size match for the slices between the
> two vendors.
> 
> Now here's the problem...
> 
> Let's posit D1 and D2, where D1 is from vendor 1 and D2 is from vendor 2.
> 
> There is also a "D1'" and "D2'", which are "spares" from each vendor.
> 
> I wish to be able to do the following:
> 
> 1. Run on the mirrored system normally.
> 
> 2. Attach any of the remaining two disks (D1' or D2') as a third mirror
>    component, allow it to synchronize, detach it, and put it in a safe
>    offsite.  This provides me with a simple offsite backup, although I 
>    understand that I risk data inconsistency unless I quiesce the system
>    prior to detaching the backup volume.
> 
> 3. If either D1 or D2 fail, I need to be able to replace the failed disk
>    with EITHER D1' or D2', and have the rebuild succeed.
> 
> Now here's what I see as a problem.
> 
> I have built the system on the SMALLER capacity disk (D1), using the
> maximum available size for the first (and only) slice on that disk.  
> 
> When I attempt to set up the larger capacity disk with the exact same 
> slice size in blocks as the smaller disk has, the system refuses, and 
> rounds either up or down (presumably due to an internal difference in how 
> it sees geometry - although the FDISK display only differs in the cylinder 
> count!)
> 
> Gmirror refuses to attach a slice which has FEWER blocks than the online
> volume with a cryptic ("Bad address") error message.  
> 
> This makes sense; it can't actually mirror it.
> 
> It will, however, attach a disk slice which has MORE blocks quite happily.
> 
> Now here's the problem.
> 
> Let's say that I have the system sync'd with the larger and smaller block
> count disks online.  The SMALLER one fails.  I replace it, but the other 
> larger disk is not available, because its in the vault.  I thus replace the
> disk with the SMALLER one, but I can't attach the slice on the replacement 
> disk, because that disk has a smaller capacity than the only remaining
> active mirror component, and I can't make a slice bigger than the number 
> of actual blocks that exist on the drive!
> 
> I'm screwed, and have no way out of this other than to ditch the entire
> scheme, manually recreate the smaller volume, copy all the data over, and
> restart gmirror from scratch.  
> 
> There goes my online mirroring - the very intent of the mirroring has been 
> lost!
> 
> It appears that you have to somehow force FDISK to create a slice with 
> the EXACT same block count to resolve this.  
> 
> This, however, appears to be flatly impossible to coerce FDISK to do, as 
> whatever is preventing this from being done is not "exposed" and visible.
> 
> There has to be a way around this problem - its not credible to insist that
> someone buy the exact same model and make of disks all the time, is it?
> 
> So... how do you work around this - or can you?
> 
> --
> -- 
> Karl Denninger (karl at denninger.net) Internet Consultant & Kids Rights Activist
> http://www.denninger.net	My home on the net - links to everything I do!
> http://scubaforum.org		Your UNCENSORED place to talk about DIVING!
> http://www.spamcuda.net		SPAM FREE mailboxes - FREE FOR A LIMITED TIME!
> http://genesis3.blogspot.com	Musings Of A Sentient Mind
> 
> 
> _______________________________________________
> freebsd-geom at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-geom
> To unsubscribe, send any mail to "freebsd-geom-unsubscribe at freebsd.org"
> 
> 
> %SPAMBLOCK-SYS: Matched [freebsd], message ok