whole-disk gmirror and upgrade from 8.2 to 9.0

[Vick Khera]

Earlier on questions@ I asked about the gmirror failure on booting into a 9.0 kernel when the whole-disk was mirrored on an 8.2 system.  I was pointed to this post <http://lists.freebsd.org/pipermail/freebsd-geom/2011-October/005071.html>, which helped me get it to boot, and I verified that I have the exact same issue mentioned there with the difference in gpart.

So I figured what I should do is destroy my existing gmirror while booted in 8.2, boot into 9.0 and install world.  This went extremely well.... that is until I try to recreate the new gmirror.

I destroyed the gmirror in 8.2 with these commands:

gmirror remove gm0 ad6
reboot
gmirror remove gm0 ad4

reboot into 9.0 kernel, install world, merge master, etc.

So now that my ada0 (old ad4) disk has on it a full 9.0 install, I want to set up gmirror again, like I always do:

boot to single user, then:

# uname -a
FreeBSD  9.0-RELEASE FreeBSD 9.0-RELEASE #0: Tue Jan 17 14:49:58 EST 2012     vivek at lorax.kcilink.com:/u/lorax1/usr9/obj.amd64/u/lorax1/usr9/src/sys/KCI64  amd64
# gmirror list
# gmirror status
# gmirror dump ada1
Can't read metadata from ada1: Invalid argument.
gmirror: Not fully done.
# gmirror dump ada0
Can't read metadata from ada0: Invalid argument.
gmirror: Not fully done.
# sysctl kern.geom.debugflags=16
kern.geom.debugflags: 0 -> 16
# gmirror label -v gm0 ada0
Metadata value stored on ada0.
Done.
# reboot

On reboot, I again get the same "GEOM_PART: integrity check failed (mirror/gm0, MBR)" error from before, and then the system proceeds to boot the ada1 disk, giving me 9.0 kernel with 8.2 world.  gmirror reports the mirror is fine, though:

# gmirror status
      Name    Status  Components
mirror/gm0  COMPLETE  ada0
# gmirror list  
Geom name: gm0
State: COMPLETE
Components: 1
Balance: load
Slice: 4096
Flags: NONE
GenID: 0
SyncID: 1
ID: 168727457
Providers:
1. Name: mirror/gm0
   Mediasize: 32016530944 (30G)
   Sectorsize: 512
   Mode: r0w0e0
Consumers:
1. Name: ada0
   Mediasize: 32016531456 (30G)
   Sectorsize: 512
   Mode: r1w1e1
   State: ACTIVE
   Priority: 0
   Flags: NONE
   GenID: 0
   SyncID: 1
   ID: 2335861329


If I set kern.geom.part.check_integrity=0 on the boot, I boot fine on the gm0 device, and gmirror reports the same status "COMPLETE".  If I clear the gmirror by removing ada0, it will boot and run fine from the ada0 device.


What do I need to do to make this gmirror work again?  The disk is partitioned MBR so there should be no problems with GPT interference.