Mirrors and Stripes and Bad Disks, Oh My

[Tony Holmes]

Hi All,

I have a system set up with 4 disks ad4/ad6/ad8/ad10 in a combination of 
RAID 1 and RAID 10.

ad4/ad8 and ad6/ad10 are mirrored (gm0 and gm1 respectively). I have set
up mirrored disks as per

http://onlamp.com/pub/a/bsd/2005/11/10/FreeBSD_Basics.html

I have put 4 partitions on both pairs. The first 4 partitions of each are
/, /var, /usr, /tmp.

The last partition on both mirror sets are striped and used for /usr/home.

Ok, ad8 dropped out - no biggie but I didn't notice it for a couple days.
On Resync, ad4 has hard read errors. It rebooted in the process, marking
ad8 as the master and ad4 as dirty (when the ad4->ad8 resynch failed!!)
ad8 does need to be replaced as well as it has flaked out after a long
period of time (could just be power/cable, but I don't trust it).

So now I'm up a river a little and could use some guidance.

I need to preserve the data on the stripe, swap out ad8 and ad4. 

I know that geom metadata is indepedant of the partition data. I need to 
get ad4 back as the master in the mirror without losing actual data. So
I was thinking of doing the following - and geom/disk experts please jump in
if you see any problem:

1. Bring up the system in single user mode.
2. Destroy gm0 (with ad4 and ad8)
3. Recreate gm0 with ad4 only
4. Break ad10 out of the gm1 mirror pair - this way during recovery I 
   can build another copy of the stripe using ad8/ad10 if I need it. 
   I expect to hit bad sectors on the copy off. I also do expect lost
   files/data if I have to go this route due to the time that ad8 was not
   mirrored.

Will this give me gm0 with a master ad4?
If I am able to recreate the stripe with identical parameters, will the
file system be visible.

It's a pita and yes I can recover most from backups but not a couple
critical files that went on today (of course).

TIA for any assistance.

-- 
Tony Holmes

Ph: (416) 993-1219

Founder and Senior Systems Architect
Crosswinds Internet Communications Inc.