configuration/performance question

Daniel P. Zepeda freebsd at zepeda.ws
Mon Jan 17 06:46:15 PST 2005


Hello,

My original problem was that I was having the problem where my mirror comes
up DEGRADED when I reboot the machine. Now, I wrote up this long post to
describe the situation and appeal for help from those folks more 
intimate with
all this than I, and, on going to subscribe to the list to post this, I 
found
the solution on digging further in the archive. I've left the whole thing
intact here, but I found the solution to my problem here;

http://lists.freebsd.org/pipermail/freebsd-geom/2004-September/000259.html

But what I uncovered while having my problem is that if I shut down one
consumer, the newer one I configured that I wanted to work, namely da1 
on gm0,
that an older mirror that I first configured, namely gm0s1, starts up, with
consumers that I originally set up as da1s1*  !!

My question now is, I have some idea of how GEOM works from the manpage, it
seems to say that the disk data flows through a set of geoms to/from the
kernel/disks. Will the "ghost" mirror and slices provide extra steps, and
therefore a performance slowdown, or am I totally off base with the way GEOM
works? (Which is likely since I was only introduced to it last week). 
That is,
is the way it is set up now on my box Ok? It works Ok, and I can reboot now
with a COMPLETE array, but is it messed up internally? I haven't done any
performance tests yet, so I don't know.

Can anyone enlighten me on this?

Here is the text of the original message. I'm leaving it here because it
provides a lot of detail on my set up to back up my question above, although
the problem I outlined below is SOLVED and my question is above.

Thanks,
DZ

******************************************************************************************


I've completely turned off swap for now, but I still swapoff="YES" in my
/etc/rc.conf, and I've consistently used shutdown since I learned about the
swap consequences.

I'm  running:

FreeBSD localhost 5.3-RELEASE-p4 FreeBSD 5.3-RELEASE-p4 #6: Fri Jan 14 
06:42:28 CST 2005    toor at localhost:/usr/obj/usr/src/sys/MYKERNEL  i386

freshly CVSup'd on a Dell PowerEdge 2850 with this patch:

http://lists.freebsd.org/pipermail/freebsd-geom/2004-October/000338.html

with the delay set to 10 as the poster originally stated. I originally 
started
with the approach #2, mirror-on-a-slice, but encountered problems because on
bootup GEOM couldn't update the metadata on da0. So I went with Approach 
#1 in
the article, and I no longer see the problem with writing metadata, but 
now it
seems that it comes up DEGRADED because there is other information left over
from the original setup. When I remove da1 from the mirror the old gm0s1
mirror starts up, although I've tried every way I can think of to get rid of
it. When I remove the old mirror that is starting up, gm0s1, then
kern.geom.conftx shows da1s1 MBR's and partitions.  How do I get rid
them?  I've tried very many methods, even  going back to
scratch, removing gmirror and just booting of the /dev/da* devices, and 
doing
it all over again. I've even tried zeroing each disk with dd and 
reconnecting them,
but to no avail. I'm thinking that this is part of the problem preventing my
mirror from coming up complete after a reboot.

I've been through the freebsd-GEOM archives, but I honestly don't know what
has been folded into what I'm running now, and what other patches to try, if
that is even the problem.

I've prepared a complete log of how it looks COMPLETE, and then after the
reboot and rebuild.

I've really wasted a good bit of time on this, I would appreciate any help,
the final goal is to have a COMPLETE mirror through reboots.

*****************************************************************************************
Here we go, Here after a rebuild, the mirror is complete and happy.
*****************************************************************************************


gmirror list
Geom name: gm0
State: COMPLETE
Components: 2
Balance: round-robin
Slice: 4096
Flags: NONE
SyncID: 2
ID: 2505058140
Providers:
1. Name: mirror/gm0
   Mediasize: 36420074496 (34G)
   Sectorsize: 512
   Mode: r1w1e2
Consumers:
1. Name: da0
   Mediasize: 36420075008 (34G)
   Sectorsize: 512
   Mode: r1w1e3
   State: ACTIVE
   Priority: 0
   Flags: DIRTY
   SyncID: 2
   ID: 4003588011
2. Name: da1
   Mediasize: 36420075008 (34G)
   Sectorsize: 512
   Mode: r1w1e3
   State: ACTIVE
   Priority: 0
   Flags: DIRTY
   SyncID: 2
   ID: 3220781586

Geom name: gm0.sync


*****************************************************************************************
********** REBOOT 
Now we're logged in again, we see the mirror in DEGRADED mode, rebuilding,
Here's a listing:
*****************************************************************************************


root at localhost:/root# gmirror list
Geom name: gm0
State: DEGRADED
Components: 2
Balance: round-robin
Slice: 4096
Flags: NONE
SyncID: 3
ID: 2505058140
Providers:
1. Name: mirror/gm0
   Mediasize: 36420074496 (34G)
   Sectorsize: 512
   Mode: r2w1e2
Consumers:
1. Name: da0
   Mediasize: 36420075008 (34G)
   Sectorsize: 512
   Mode: r2w1e3
   State: ACTIVE
   Priority: 0
   Flags: DIRTY
   SyncID: 3
   ID: 4003588011
2. Name: da1
   Mediasize: 36420075008 (34G)
   Sectorsize: 512
   Mode: r0w1e1
   State: SYNCHRONIZING
   Priority: 0
   Flags: DIRTY, SYNCHRONIZING
   SyncID: 3
   Synchronized: 17%
   ID: 3220781586

Geom name: gm0.sync
Consumers:
1. Name: mirror/gm0
   Mediasize: 36420074496 (34G)
   Sectorsize: 512
   Mode: r1w0e0

*****************************************************************************************
*************
****** What kern.geom.conftx during the rebuild
********
*****************************************************************************************

# sysctl -b kern.geom.conftx
0 DISK da1 36420075008 512 hd 255 sc 63
1 MIRROR mirror/gm0 36420074496 512
2 MBR mirror/gm0s1 36413282304 512 i 0 o 32256 ty 165
3 BSD mirror/gm0s1c 36413282304 512 i 2 o 0 ty 0
3 BSD mirror/gm0s1b 1073733632 512 i 1 o 35339548672 ty 1
3 BSD mirror/gm0s1a 35339540480 512 i 0 o 8192 ty 7
0 DISK da0 36420075008 512 hd 255 sc 63
1 MIRROR mirror/gm0 36420074496 512
2 MBR mirror/gm0s1 36413282304 512 i 0 o 32256 ty 165
3 BSD mirror/gm0s1c 36413282304 512 i 2 o 0 ty 0
3 BSD mirror/gm0s1b 1073733632 512 i 1 o 35339548672 ty 1
3 BSD mirror/gm0s1a 35339540480 512 i 0 o 8192 ty 7

*****************************************************************************************
*******************
*******  And here's /var/log/messages from bootup, until this point
******************
*****************************************************************************************




Jan 17 06:43:55 localhost kernel: da0 at mpt0 bus 0 target 0 lun 0
Jan 17 06:43:55 localhost kernel: da0: <SEAGATE ST336607LC DS09> Fixed 
Direct Access SCSI-3 device
Jan 17 06:43:55 localhost kernel: da0: 320.000MB/s transfers 
(160.000MHz, offset 63, 16bit), Tagged Queueing Enabled
Jan 17 06:43:55 localhost kernel: da0: 34732MB (71132959 512 byte 
sectors: 255H 63S/T 4427C)
Jan 17 06:43:55 localhost kernel: da1 at mpt0 bus 0 target 1 lun 0
Jan 17 06:43:55 localhost kernel: da1: <SEAGATE ST336607LC DS09> Fixed 
Direct Access SCSI-3 device
Jan 17 06:43:55 localhost kernel: da1: 320.000MB/s transfers 
(160.000MHz, offset 63, 16bit), Tagged Queueing Enabled
Jan 17 06:43:55 localhost kernel: da1: 34732MB (71132959 512 byte 
sectors: 255H 63S/T 4427C)
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Tasting da0.
Jan 17 06:43:55 localhost kernel: magic: GEOM::MIRROR
Jan 17 06:43:55 localhost kernel: version: 1
Jan 17 06:43:55 localhost kernel: name: gm0
Jan 17 06:43:55 localhost kernel: mid: 2505058140
Jan 17 06:43:55 localhost kernel: did: 4003588011
Jan 17 06:43:55 localhost kernel: all: 2
Jan 17 06:43:55 localhost kernel: syncid: 2
Jan 17 06:43:55 localhost kernel: priority: 0
Jan 17 06:43:55 localhost kernel: slice: 4096
Jan 17 06:43:55 localhost kernel: balance: round-robin
Jan 17 06:43:55 localhost kernel: mediasize: 36420074496
Jan 17 06:43:55 localhost kernel: sectorsize: 512
Jan 17 06:43:55 localhost kernel: syncoffset: 0
Jan 17 06:43:55 localhost kernel: mflags: NONE
Jan 17 06:43:55 localhost kernel: dflags: NONE
Jan 17 06:43:55 localhost kernel: hcprovider:
Jan 17 06:43:55 localhost kernel: MD5 hash: 05b629f9060ab32a0d273902be366a37
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[1]: Creating device gm0 
(id=2505058140).
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[0]: Device gm0 created 
(id=2505058140).
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[1]: Adding disk da0 to gm0.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Adding disk da0.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Disk da0 connected.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[1]: Disk da0 state changed 
from NONE to NEW (device gm0).
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[0]: Device gm0: provider 
da0 detected.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Tasting da1.
Jan 17 06:43:55 localhost kernel: magic: GEOM::MIRROR
Jan 17 06:43:55 localhost kernel: version: 1
Jan 17 06:43:55 localhost kernel: name: gm0
Jan 17 06:43:55 localhost kernel: mid: 2505058140
Jan 17 06:43:55 localhost kernel: did: 3220781586
Jan 17 06:43:55 localhost kernel: all: 2
Jan 17 06:43:55 localhost kernel: syncid: 2
Jan 17 06:43:55 localhost kernel: priority: 0
Jan 17 06:43:55 localhost kernel: slice: 4096
Jan 17 06:43:55 localhost kernel: balance: round-robin
Jan 17 06:43:55 localhost kernel: mediasize: 36420074496
Jan 17 06:43:55 localhost kernel: sectorsize: 512
Jan 17 06:43:55 localhost kernel: syncoffset: 0
Jan 17 06:43:55 localhost kernel: mflags: NONE
Jan 17 06:43:55 localhost kernel: dflags: NONE
Jan 17 06:43:55 localhost kernel: hcprovider:
Jan 17 06:43:55 localhost kernel: MD5 hash: b4ab3313a1ead3ed591d0b3f4deed554
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[1]: Adding disk da1 to gm0.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Adding disk da1.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Disk da1 connected.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[1]: Disk da1 state changed 
from NONE to NEW (device gm0).
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[0]: Device gm0: provider 
da1 detected.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[1]: Device gm0 state 
changed from STARTING to RUNNING.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[1]: Disk da1 state changed 
from NEW to ACTIVE (device gm0).
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r0w1e1 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r0w-1e-1 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Metadata on da1 updated.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[0]: Device gm0: provider 
da1 activated.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[1]: Disk da0 state changed 
from NEW to ACTIVE (device gm0).
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r0w1e1 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Tasting da0s1.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Tasting da1s1.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r0w-1e-1 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Metadata on da0 updated.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[0]: Device gm0: provider 
da0 activated.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[0]: Device gm0: provider 
mirror/gm0 launched.
Jan 17 06:43:55 localhost kernel: magic: GEOM::MIRROR
Jan 17 06:43:55 localhost kernel: version: 1
Jan 17 06:43:55 localhost kernel: name: gm0s1
Jan 17 06:43:55 localhost kernel: mid: 3061091415
Jan 17 06:43:55 localhost kernel: did: 3644757297
Jan 17 06:43:55 localhost kernel: all: 2
Jan 17 06:43:55 localhost kernel: syncid: 17
Jan 17 06:43:55 localhost kernel: priority: 0
Jan 17 06:43:55 localhost kernel: slice: 4096
Jan 17 06:43:55 localhost kernel: balance: round-robin
Jan 17 06:43:55 localhost kernel: mediasize: 36413281792
Jan 17 06:43:55 localhost kernel: sectorsize: 512
Jan 17 06:43:55 localhost kernel: syncoffset: 0
Jan 17 06:43:55 localhost kernel: mflags: NONE
Jan 17 06:43:55 localhost kernel: dflags: NONE
Jan 17 06:43:55 localhost kernel: hcprovider: da1s1
Jan 17 06:43:55 localhost kernel: MD5 hash: a3fe1554ce5cb9364934713102300b18
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[1]: Creating device gm0s1 
(id=3061091415).
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[0]: Device gm0s1 created 
(id=3061091415).
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[1]: Adding disk da1s1 to 
gm0s1.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Adding disk da1s1.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Disk da1s1 connected.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[1]: Disk da1s1 state 
changed from NONE to NEW (device gm0s1).
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[0]: Device gm0s1: provider 
da1s1 detected.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r1w0e0.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r1w0e1 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w0e1 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r-1w0e0.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r-1w0e-1 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r-1w0e-1 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Tasting mirror/gm0.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r1w0e0.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r1w0e1 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w0e1 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r-1w0e0.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r-1w0e-1 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r-1w0e-1 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r1w0e0.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r1w0e1 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w0e1 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r-1w0e0.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r-1w0e-1 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r-1w0e-1 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r1w0e0.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r1w0e1 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w0e1 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r-1w0e0.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r-1w0e-1 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r-1w0e-1 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Tasting da1s1a.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Tasting da1s1b.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Tasting da1s1c.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r1w0e1.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r1w0e2 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w0e2 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r-1w0e-1.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r-1w0e-2 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r-1w0e-2 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Tasting mirror/gm0s1.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r1w0e1.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r1w0e2 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w0e2 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r-1w0e-1.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r-1w0e-2 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r-1w0e-2 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r1w0e1.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r1w0e2 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w0e2 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r-1w0e-1.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r-1w0e-2 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r-1w0e-2 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r1w0e1.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r1w0e2 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w0e2 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r-1w0e-1.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r-1w0e-2 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r-1w0e-2 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r1w0e1.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r1w0e2 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w0e2 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r-1w0e-1.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r-1w0e-2 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r-1w0e-2 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r1w0e2.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r1w0e3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w0e3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r-1w0e-2.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r-1w0e-3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r-1w0e-3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Tasting mirror/gm0s1a.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r1w0e2.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r1w0e3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w0e3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r-1w0e-2.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r-1w0e-3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r-1w0e-3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r1w0e2.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r1w0e3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w0e3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r-1w0e-2.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r-1w0e-3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r-1w0e-3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r1w0e2.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r1w0e3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w0e3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r-1w0e-2.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r-1w0e-3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r-1w0e-3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r1w0e2.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r1w0e3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w0e3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r-1w0e-2.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r-1w0e-3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r-1w0e-3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Tasting mirror/gm0s1b.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r1w0e2.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r1w0e3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w0e3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r-1w0e-2.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r-1w0e-3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r-1w0e-3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r1w0e2.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r1w0e3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w0e3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r-1w0e-2.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r-1w0e-3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r-1w0e-3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r1w0e2.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r1w0e3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w0e3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r-1w0e-2.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r-1w0e-3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r-1w0e-3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r1w0e2.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r1w0e3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w0e3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r-1w0e-2.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r-1w0e-3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r-1w0e-3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Tasting mirror/gm0s1c.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r1w0e2.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r1w0e3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w0e3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r-1w0e-2.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r-1w0e-3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r-1w0e-3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r1w0e2.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r1w0e3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w0e3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r-1w0e-2.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r-1w0e-3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r-1w0e-3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r1w0e2.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r1w0e3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w0e3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r-1w0e-2.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r-1w0e-3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r-1w0e-3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[0]: Force device gm0s1 
start due to timeout.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[1]: Device gm0s1 state 
changed from STARTING to RUNNING.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[1]: Disk da1s1 state 
changed from NEW to ACTIVE (device gm0s1).
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r0w1e1 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[1]: Disk da1 state changed 
from ACTIVE to DISCONNECTED (device gm0).
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[0]: Device gm0: provider 
da1 disconnected.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Disk da1 disconnected.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Consumer da1 destroyed.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[1]: Device gm0: syncid 
bumped to 3.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r0w1e1 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r0w-1e-1 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Metadata on da1s1 updated.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[0]: Device gm0s1: provider 
da1s1 activated.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[0]: Device gm0s1: provider 
mirror/gm0s1 launched.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r0w-1e-1 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Metadata on da0 updated.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Tasting da1.
Jan 17 06:43:55 localhost kernel: magic: GEOM::MIRROR
Jan 17 06:43:55 localhost kernel: version: 1
Jan 17 06:43:55 localhost kernel: name: gm0
Jan 17 06:43:55 localhost kernel: mid: 2505058140
Jan 17 06:43:55 localhost kernel: did: 3220781586
Jan 17 06:43:55 localhost kernel: all: 2
Jan 17 06:43:55 localhost kernel: syncid: 2
Jan 17 06:43:55 localhost kernel: priority: 0
Jan 17 06:43:55 localhost kernel: slice: 4096
Jan 17 06:43:55 localhost kernel: balance: round-robin
Jan 17 06:43:55 localhost kernel: mediasize: 36420074496
Jan 17 06:43:55 localhost kernel: sectorsize: 512
Jan 17 06:43:55 localhost kernel: syncoffset: 0
Jan 17 06:43:55 localhost kernel: mflags: NONE
Jan 17 06:43:55 localhost kernel: dflags: NONE
Jan 17 06:43:55 localhost kernel: hcprovider:
Jan 17 06:43:55 localhost kernel: MD5 hash: 7a64c716e9497f74a61b8dc8be0dad3f
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[1]: Adding disk da1 to gm0.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Adding disk da1.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Disk da1 connected.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[1]: Disk da1 state changed 
from NONE to NEW (device gm0).
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[0]: Device gm0: provider 
da1 detected.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[1]: Disk da1 state changed 
from NEW to SYNCHRONIZING (device gm0).
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[0]: Device gm0: rebuilding 
provider da1.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r0w1e1 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r1w0e0.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w0e1 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Metadata on da1 updated.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0s1: r1w0e0.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r1w0e1 = 6
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Tasting mirror/gm0s1.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0s1: r1w0e0.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r1w0e1 = 6
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0s1: r1w0e0.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r1w0e1 = 6
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0s1: r1w0e0.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r1w0e1 = 6
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[1]: Disk da1s1 state 
changed from ACTIVE to DISCONNECTED (device gm0s1).
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[0]: Device gm0s1: provider 
da1s1 disconnected.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Disk da1s1 disconnected.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Consumer da1s1 destroyed.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: No I/O requests for 
gm0s1, it can be destroyed.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[0]: Device gm0s1: provider 
mirror/gm0s1 destroyed.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[0]: Device gm0s1 destroyed.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Tasting da0s1.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Tasting da0s1a.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Tasting da0s1b.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Tasting da0s1c.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r1w1e2.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w1e2 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[1]: Disk da0 (device gm0) 
marked as dirty.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Metadata on da0 updated.
Jan 17 06:43:55 localhost kernel: Mounted root from ufs:/dev/mirror/gm0s1a.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Metadata on da1 updated.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r1w0e0.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w0e0 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r-1w0e0.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r-1w0e0 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r1w0e0.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w0e0 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r0w1e0.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r0w1e0 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r-1w0e0.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r-1w0e0 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r0w-1e0.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r0w-1e0 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Metadata on da1 updated.
Jan 17 06:45:09 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r1w0e0.
Jan 17 06:45:09 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w0e0 = 0
Jan 17 06:45:09 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r-1w0e0.
Jan 17 06:45:09 localhost kernel: GEOM_MIRROR[2]: Access da0 r-1w0e0 = 0
Jan 17 06:45:10 localhost kernel: GEOM_MIRROR[2]: Metadata on da1 updated.
Jan 17 06:53:07 localhost last message repeated 26 times



*****************************************************************************************
********************
***** Now I disconnect da1 from gm0 I originally did this because I 
wanted to
just zero /dev/da1 with dd and start over (or so I thought). But when I 
ran a
list, I was surprised to see gm0s1 starting up:

************************
*****************************************************************************************
root at localhost:/root# gmirror deactivate gm0 da1
root at localhost:/root# gmirror list
Geom name: gm0
State: DEGRADED
Components: 2
Balance: round-robin
Slice: 4096
Flags: NONE
SyncID: 4
ID: 2505058140
Providers:
1. Name: mirror/gm0
   Mediasize: 36420074496 (34G)
   Sectorsize: 512
   Mode: r1w1e2
Consumers:
1. Name: da0
   Mediasize: 36420075008 (34G)
   Sectorsize: 512
   Mode: r1w1e3
   State: ACTIVE
   Priority: 0
   Flags: DIRTY
   SyncID: 4
   ID: 4003588011

Geom name: gm0.sync

Geom name: gm0s1
State: DEGRADED
Components: 2
Balance: round-robin
Slice: 4096
Flags: NONE
SyncID: 17
ID: 3061091415
Providers:
1. Name: mirror/gm0s1
   Mediasize: 36413281792 (34G)
   Sectorsize: 512
   Mode: r0w0e0
Consumers:
1. Name: da1s1
   Mediasize: 36413282304 (34G)
   Sectorsize: 512
   Mode: r0w0e0
   State: ACTIVE
   Priority: 0
   Flags: HARDCODED
   SyncID: 17
   ID: 3644757297

Geom name: gm0s1.sync

*****************************************************************************************
**********************************
**** Now kern.geom.conftx looked like this
*****************************************************************************************
*************************

# sysctl -b kern.geom.conftx
0 DISK da1 36420075008 512 hd 255 sc 63
1 MBR da1s1 36413282304 512 i 0 o 32256 ty 165
2 BSD da1s1c 36413282304 512 i 2 o 0 ty 0
2 BSD da1s1b 1073733632 512 i 1 o 35339548672 ty 1
2 BSD da1s1a 35339540480 512 i 0 o 8192 ty 7
2 MIRROR mirror/gm0s1 36413281792 512
3 BSD mirror/gm0s1c 36413282304 512 i 2 o 0 ty 0
3 BSD mirror/gm0s1b 1073733632 512 i 1 o 35339548672 ty 1
3 BSD mirror/gm0s1a 35339540480 512 i 0 o 8192 ty 7
0 DISK da0 36420075008 512 hd 255 sc 63
1 MIRROR mirror/gm0 36420074496 512
2 MBR mirror/gm0s1 36413282304 512 i 0 o 32256 ty 165
3 BSD mirror/gm0s1c 36413282304 512 i 2 o 0 ty 0
3 BSD mirror/gm0s1b 1073733632 512 i 1 o 35339548672 ty 1
3 BSD mirror/gm0s1a 35339540480 512 i 0 o 8192 ty 7

*****************************************************************************************
***********************************
**** /var/log/messages for this event:
**********************************
*****************************************************************************************

Jan 17 06:53:07 localhost kernel: GEOM_MIRROR[1]: Disk da1 state changed 
from SYNCHRONIZING to DISCONNECTED (device gm0).
Jan 17 06:53:07 localhost kernel: GEOM_MIRROR[0]: Device gm0: provider 
da1 disconnected.
Jan 17 06:53:07 localhost kernel: GEOM_MIRROR[0]: Device gm0: rebuilding 
provider da1 stopped.
Jan 17 06:53:07 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r-1w0e0.
Jan 17 06:53:07 localhost kernel: GEOM_MIRROR[2]: Access da0 r-1w0e0 = 0
Jan 17 06:53:07 localhost kernel: GEOM_MIRROR[2]: Consumer mirror/gm0 
destroyed.
Jan 17 06:53:07 localhost kernel: GEOM_MIRROR[2]: Access da1 r0w-1e-1 = 0
Jan 17 06:53:07 localhost kernel: GEOM_MIRROR[2]: Disk da1 disconnected.
Jan 17 06:53:07 localhost kernel: GEOM_MIRROR[2]: I/O requests for da1 
in queue, can't destroy it now.
Jan 17 06:53:07 localhost kernel: GEOM_MIRROR[2]: Consumer da1 destroyed.
Jan 17 06:53:07 localhost kernel: GEOM_MIRROR[2]: Tasting da1.
Jan 17 06:53:07 localhost kernel: GEOM_MIRROR[0]: Device gm0: provider 
da1 marked as inactive, skipping.
Jan 17 06:53:07 localhost kernel: GEOM_MIRROR[2]: Tasting da1s1.
Jan 17 06:53:07 localhost kernel: magic: GEOM::MIRROR
Jan 17 06:53:07 localhost kernel: version: 1
Jan 17 06:53:07 localhost kernel: name: gm0s1
Jan 17 06:53:07 localhost kernel: mid: 3061091415
Jan 17 06:53:07 localhost kernel: did: 3644757297
Jan 17 06:53:07 localhost kernel: all: 2
Jan 17 06:53:07 localhost kernel: syncid: 17
Jan 17 06:53:07 localhost kernel: priority: 0
Jan 17 06:53:07 localhost kernel: slice: 4096
Jan 17 06:53:07 localhost kernel: balance: round-robin
Jan 17 06:53:07 localhost kernel: mediasize: 36413281792
Jan 17 06:53:07 localhost kernel: sectorsize: 512
Jan 17 06:53:07 localhost kernel: syncoffset: 0
Jan 17 06:53:07 localhost kernel: mflags: NONE
Jan 17 06:53:07 localhost kernel: dflags: NONE
Jan 17 06:53:07 localhost kernel: hcprovider: da1s1
Jan 17 06:53:07 localhost kernel: MD5 hash: 67a736074371d7996a699af47c6fb79c
Jan 17 06:53:07 localhost kernel: GEOM_MIRROR[1]: Creating device gm0s1 
(id=3061091415).
Jan 17 06:53:07 localhost kernel: GEOM_MIRROR[0]: Device gm0s1 created 
(id=3061091415).
Jan 17 06:53:07 localhost kernel: GEOM_MIRROR[1]: Adding disk da1s1 to 
gm0s1.
Jan 17 06:53:07 localhost kernel: GEOM_MIRROR[2]: Adding disk da1s1.
Jan 17 06:53:07 localhost kernel: GEOM_MIRROR[2]: Disk da1s1 connected.
Jan 17 06:53:07 localhost kernel: GEOM_MIRROR[1]: Disk da1s1 state 
changed from NONE to NEW (device gm0s1).
Jan 17 06:53:07 localhost kernel: GEOM_MIRROR[0]: Device gm0s1: provider 
da1s1 detected.
Jan 17 06:53:07 localhost kernel: GEOM_MIRROR[2]: Tasting da1s1a.
Jan 17 06:53:07 localhost kernel: GEOM_MIRROR[1]: Device gm0: syncid 
bumped to 4.
Jan 17 06:53:07 localhost kernel: GEOM_MIRROR[2]: Metadata on da0 updated.
Jan 17 06:53:07 localhost kernel: GEOM_MIRROR[2]: Tasting da1s1b.
Jan 17 06:53:07 localhost kernel: GEOM_MIRROR[2]: Tasting da1s1c.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[0]: Force device gm0s1 
start due to timeout.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[1]: Device gm0s1 state 
changed from STARTING to RUNNING.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[1]: Disk da1s1 state 
changed from NEW to ACTIVE (device gm0s1).
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r0w1e1 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r0w-1e-1 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Metadata on da1s1 updated.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[0]: Device gm0s1: provider 
da1s1 activated.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[0]: Device gm0s1: provider 
mirror/gm0s1 launched.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Tasting da1.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[0]: Device gm0: provider 
da1 marked as inactive, skipping.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0s1: r1w0e0.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r1w0e1 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0s1: r-1w0e0.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r-1w0e-1 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Tasting mirror/gm0s1.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0s1: r1w0e0.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r1w0e1 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0s1: r-1w0e0.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r-1w0e-1 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0s1: r1w0e0.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r1w0e1 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0s1: r-1w0e0.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r-1w0e-1 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0s1: r1w0e0.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r1w0e1 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0s1: r-1w0e0.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r-1w0e-1 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Tasting da1s1a.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Tasting da1s1b.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Tasting da1s1c.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0s1: r1w0e1.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r1w0e2 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0s1: r-1w0e-1.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r-1w0e-2 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Tasting mirror/gm0s1a.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0s1: r1w0e1.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r1w0e2 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0s1: r-1w0e-1.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r-1w0e-2 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0s1: r1w0e1.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r1w0e2 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0s1: r-1w0e-1.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r-1w0e-2 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0s1: r1w0e1.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r1w0e2 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0s1: r-1w0e-1.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r-1w0e-2 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0s1: r1w0e1.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r1w0e2 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0s1: r-1w0e-1.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r-1w0e-2 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Tasting mirror/gm0s1b.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0s1: r1w0e1.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r1w0e2 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0s1: r-1w0e-1.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r-1w0e-2 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0s1: r1w0e1.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r1w0e2 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0s1: r-1w0e-1.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r-1w0e-2 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0s1: r1w0e1.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r1w0e2 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0s1: r-1w0e-1.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r-1w0e-2 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0s1: r1w0e1.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r1w0e2 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0s1: r-1w0e-1.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r-1w0e-2 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Tasting mirror/gm0s1c.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0s1: r1w0e1.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r1w0e2 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0s1: r-1w0e-1.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r-1w0e-2 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0s1: r1w0e1.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r1w0e2 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0s1: r-1w0e-1.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r-1w0e-2 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0s1: r1w0e1.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r1w0e2 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0s1: r-1w0e-1.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r-1w0e-2 = 0



*****************************************************************************************
*************************
********* forgetting on gm0s1 so I can remove it
**************************
*****************************************************************************************

root at localhost:/root# gmirror  forget gm0s1
root at localhost:/root# gmirror list
Geom name: gm0
State: DEGRADED
Components: 2
Balance: round-robin
Slice: 4096
Flags: NONE
SyncID: 4
ID: 2505058140
Providers:
1. Name: mirror/gm0
   Mediasize: 36420074496 (34G)
   Sectorsize: 512
   Mode: r1w1e2
Consumers:
1. Name: da0
   Mediasize: 36420075008 (34G)
   Sectorsize: 512
   Mode: r1w1e3
   State: ACTIVE
   Priority: 0
   Flags: DIRTY
   SyncID: 4
   ID: 4003588011

Geom name: gm0.sync

Geom name: gm0s1
State: COMPLETE
Components: 1
Balance: round-robin
Slice: 4096
Flags: NONE
SyncID: 17
ID: 3061091415
Providers:
1. Name: mirror/gm0s1
   Mediasize: 36413281792 (34G)
   Sectorsize: 512
   Mode: r0w0e0
Consumers:
1. Name: da1s1
   Mediasize: 36413282304 (34G)
   Sectorsize: 512
   Mode: r0w0e0
   State: ACTIVE
   Priority: 0
   Flags: HARDCODED
   SyncID: 17
   ID: 3644757297

Geom name: gm0s1.sync

*****************************************************************************************
***********************
********** So, now I want to remove gm0s1, but I can't because it always
********** comes back, I need to manually destroy it each time and then add
da1 to gm0
*****************************************************************************************
**************************

root at localhost:/root# gmirror remove gm0s1 da1s1


*************************
0 DISK da1 36420075008 512 hd 255 sc 63
1 MBR da1s1 36413282304 512 i 0 o 32256 ty 165
2 MIRROR mirror/gm0s1 36413281792 512
3 BSD mirror/gm0s1c 36413282304 512 i 2 o 0 ty 0
3 BSD mirror/gm0s1b 1073733632 512 i 1 o 35339548672 ty 1
3 BSD mirror/gm0s1a 35339540480 512 i 0 o 8192 ty 7
0 DISK da0 36420075008 512 hd 255 sc 63
1 MIRROR mirror/gm0 36420074496 512
2 MBR mirror/gm0s1 36413282304 512 i 0 o 32256 ty 165
3 BSD mirror/gm0s1c 36413282304 512 i 2 o 0 ty 0
3 BSD mirror/gm0s1b 1073733632 512 i 1 o 35339548672 ty 1
3 BSD mirror/gm0s1a 35339540480 512 i 0 o 8192 ty 7


*****************************************************************************************
**********************
********** Removing da1s1 from gm0s1 reveals the da1s1 MBR and the 
partitions
********** on da1  Huh?
*************************
*****************************************************************************************


root at localhost:/root# gmirror remove gm0s1 da1s1
root at localhost:/root# gmirror list
Geom name: gm0
State: DEGRADED
Components: 2
Balance: round-robin
Slice: 4096
Flags: NONE
SyncID: 4
ID: 2505058140
Providers:
1. Name: mirror/gm0
   Mediasize: 36420074496 (34G)
   Sectorsize: 512
   Mode: r1w1e2
Consumers:
1. Name: da0
   Mediasize: 36420075008 (34G)
   Sectorsize: 512
   Mode: r1w1e3
   State: ACTIVE
   Priority: 0
   Flags: DIRTY
   SyncID: 4
   ID: 4003588011

Geom name: gm0.sync

0 DISK da1 36420075008 512 hd 255 sc 63
1 MBR da1s1 36413282304 512 i 0 o 32256 ty 165
2 BSD da1s1c 36413282304 512 i 2 o 0 ty 0
2 BSD da1s1b 1073733632 512 i 1 o 35339548672 ty 1
2 BSD da1s1a 35339540480 512 i 0 o 8192 ty 7
0 DISK da0 36420075008 512 hd 255 sc 63
1 MIRROR mirror/gm0 36420074496 512
2 MBR mirror/gm0s1 36413282304 512 i 0 o 32256 ty 165
3 BSD mirror/gm0s1c 36413282304 512 i 2 o 0 ty 0
3 BSD mirror/gm0s1b 1073733632 512 i 1 o 35339548672 ty 1
3 BSD mirror/gm0s1a 35339540480 512 i 0 o 8192 ty 7


*****************************************************************************************
So I forget on gm0 so I can work on it,

insert da1 and I'm back in business, kern.geom.conftx looks ok now?
*****************************************************************************************



root at localhost:/root# gmirror list
Geom name: gm0
State: COMPLETE
Components: 1
Balance: round-robin
Slice: 4096
Flags: NONE
SyncID: 4
ID: 2505058140
Providers:
1. Name: mirror/gm0
   Mediasize: 36420074496 (34G)
   Sectorsize: 512
   Mode: r1w1e2
Consumers:
1. Name: da0
   Mediasize: 36420075008 (34G)
   Sectorsize: 512
   Mode: r1w1e3
   State: ACTIVE
   Priority: 0
   Flags: DIRTY
   SyncID: 4
   ID: 4003588011

Geom name: gm0.sync

root at localhost:/root# gmirror insert gm0 da1
root at localhost:/root# gmirror list
Geom name: gm0
State: DEGRADED
Components: 2
Balance: round-robin
Slice: 4096
Flags: NONE
SyncID: 4
ID: 2505058140
Providers:
1. Name: mirror/gm0
   Mediasize: 36420074496 (34G)
   Sectorsize: 512
   Mode: r2w1e2
Consumers:
1. Name: da0
   Mediasize: 36420075008 (34G)
   Sectorsize: 512
   Mode: r2w1e3
   State: ACTIVE
   Priority: 0
   Flags: DIRTY
   SyncID: 4
   ID: 4003588011
2. Name: da1
   Mediasize: 36420075008 (34G)
   Sectorsize: 512
   Mode: r0w1e1
   State: SYNCHRONIZING
   Priority: 0
   Flags: DIRTY, SYNCHRONIZING
   SyncID: 4
   Synchronized: 0%
   ID: 3782877908

Geom name: gm0.sync
Consumers:
1. Name: mirror/gm0
   Mediasize: 36420074496 (34G)
   Sectorsize: 512
   Mode: r1w0e0

0 DISK da1 36420075008 512 hd 255 sc 63
1 MIRROR mirror/gm0 36420074496 512
2 MBR mirror/gm0s1 36413282304 512 i 0 o 32256 ty 165
3 BSD mirror/gm0s1c 36413282304 512 i 2 o 0 ty 0
3 BSD mirror/gm0s1b 1073733632 512 i 1 o 35339548672 ty 1
3 BSD mirror/gm0s1a 35339540480 512 i 0 o 8192 ty 7
0 DISK da0 36420075008 512 hd 255 sc 63
1 MIRROR mirror/gm0 36420074496 512
2 MBR mirror/gm0s1 36413282304 512 i 0 o 32256 ty 165
3 BSD mirror/gm0s1c 36413282304 512 i 2 o 0 ty 0
3 BSD mirror/gm0s1b 1073733632 512 i 1 o 35339548672 ty 1
3 BSD mirror/gm0s1a 35339540480 512 i 0 o 8192 ty 7



*****************************************************************************************
And the rest of the log entries for all of this;
*****************************************************************************************



Jan 17 06:54:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r0w1e1 = 0
Jan 17 06:54:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r0w-1e-1 = 0
Jan 17 06:54:11 localhost kernel: GEOM_MIRROR[2]: Metadata on da1s1 updated.
Jan 17 06:54:11 localhost kernel: GEOM_MIRROR[2]: Tasting da1.
Jan 17 06:54:11 localhost kernel: GEOM_MIRROR[0]: Device gm0: provider 
da1 marked as inactive, skipping.
Jan 17 06:55:18 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r0w1e1 = 0
Jan 17 06:55:18 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r0w-1e-1 = 0
Jan 17 06:55:18 localhost kernel: GEOM_MIRROR[2]: Metadata on da1s1 cleared.
Jan 17 06:55:18 localhost kernel: GEOM_MIRROR[1]: Disk da1s1 state 
changed from ACTIVE to DESTROY (device gm0s1).
Jan 17 06:55:18 localhost kernel: GEOM_MIRROR[0]: Device gm0s1: provider 
da1s1 destroyed.
Jan 17 06:55:18 localhost kernel: GEOM_MIRROR[2]: Disk da1s1 disconnected.
Jan 17 06:55:18 localhost kernel: GEOM_MIRROR[2]: Consumer da1s1 destroyed.
Jan 17 06:55:18 localhost kernel: GEOM_MIRROR[2]: No I/O requests for 
gm0s1, it can be destroyed.
Jan 17 06:55:18 localhost kernel: GEOM_MIRROR[0]: Device gm0s1: provider 
mirror/gm0s1 destroyed.
Jan 17 06:55:18 localhost kernel: GEOM_MIRROR[0]: Device gm0s1 destroyed.
Jan 17 06:55:18 localhost kernel: GEOM_MIRROR[2]: Tasting da1.
Jan 17 06:55:18 localhost kernel: GEOM_MIRROR[0]: Device gm0: provider 
da1 marked as inactive, skipping.
Jan 17 06:55:18 localhost kernel: GEOM_MIRROR[2]: Tasting da1s1.
Jan 17 06:55:18 localhost kernel: GEOM_MIRROR[2]: Tasting da1s1a.
Jan 17 06:55:19 localhost kernel: GEOM_MIRROR[2]: Tasting da1s1b.
Jan 17 06:55:19 localhost kernel: GEOM_MIRROR[2]: Tasting da1s1c.
Jan 17 06:56:05 localhost kernel: GEOM_MIRROR[2]: Metadata on da0 updated.
Jan 17 06:56:22 localhost kernel: GEOM_MIRROR[2]: Metadata on da0 updated.
Jan 17 06:56:22 localhost kernel: GEOM_MIRROR[2]: Tasting da1.
Jan 17 06:56:22 localhost kernel: magic: GEOM::MIRROR
Jan 17 06:56:22 localhost kernel: version: 1
Jan 17 06:56:22 localhost kernel: name: gm0
Jan 17 06:56:22 localhost kernel: mid: 2505058140
Jan 17 06:56:22 localhost kernel: did: 3782877908
Jan 17 06:56:22 localhost kernel: all: 2
Jan 17 06:56:22 localhost kernel: syncid: 0
Jan 17 06:56:22 localhost kernel: priority: 0
Jan 17 06:56:22 localhost kernel: slice: 4096
Jan 17 06:56:22 localhost kernel: balance: round-robin
Jan 17 06:56:22 localhost kernel: mediasize: 36420074496
Jan 17 06:56:22 localhost kernel: sectorsize: 512
Jan 17 06:56:22 localhost kernel: syncoffset: 0
Jan 17 06:56:22 localhost kernel: mflags: NONE
Jan 17 06:56:22 localhost kernel: dflags: NONE
Jan 17 06:56:22 localhost kernel: hcprovider:
Jan 17 06:56:22 localhost kernel: MD5 hash: c8505c5e0fcf859952eaab3439f1b7a5
Jan 17 06:56:22 localhost kernel: GEOM_MIRROR[1]: Adding disk da1 to gm0.
Jan 17 06:56:22 localhost kernel: GEOM_MIRROR[2]: Adding disk da1.
Jan 17 06:56:22 localhost kernel: GEOM_MIRROR[2]: Disk da1 connected.
Jan 17 06:56:22 localhost kernel: GEOM_MIRROR[1]: Disk da1 state changed 
from NONE to NEW (device gm0).
Jan 17 06:56:22 localhost kernel: GEOM_MIRROR[0]: Device gm0: provider 
da1 detected.
Jan 17 06:56:22 localhost kernel: GEOM_MIRROR[1]: Disk da1 state changed 
from NEW to SYNCHRONIZING (device gm0).
Jan 17 06:56:22 localhost kernel: GEOM_MIRROR[0]: Device gm0: rebuilding 
provider da1.
Jan 17 06:56:22 localhost kernel: GEOM_MIRROR[2]: Access da1 r0w1e1 = 0
Jan 17 06:56:22 localhost kernel: GEOM_MIRROR[2]: Access request for 
mirror/gm0: r1w0e0.
Jan 17 06:56:22 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w0e0 = 0
Jan 17 06:56:22 localhost kernel: GEOM_MIRROR[2]: Metadata on da1 updated.
Jan 17 06:56:51 localhost last message repeated 28 times
Jan 17 06:56:51 localhost ntpd[452]: kernel time sync enabled 2001
Jan 17 06:56:52 localhost kernel: GEOM_MIRROR[2]: Metadata on da1 updated.
Jan 17 06:57:23 localhost last message repeated 30 times

*****************************************************************************************
Just in case, here is the contents of /dev, filtered for da*:
*****************************************************************************************

crw-r-----  1 root  operator    4,  12 Jan 17 06:43 /dev/da0
crw-r-----  1 root  operator    4,  13 Jan 17 06:43 /dev/da1




I've been banging my head on this one for awhile. Any help would be 
appreciated.

Thanks,
DZ



-------------- next part --------------
Hello, 

My original problem was that I was having the problem where my mirror comes
up DEGRADED when I reboot the machine. Now, I wrote up this long post to
describe the situation and appeal for help from those folks more intimate with
all this than I, and on going to subscribe to the list to post this, I found
the solution on digging further in the archive. I've left the whole thing
intact here, but I found the solution to my problem here;

http://lists.freebsd.org/pipermail/freebsd-geom/2004-September/000259.html

But what I uncovered while having my problem is that if I shut down one
consumer, the newer one I configured that I wanted to work, namely da1 on gm0,
that an older mirror that I first configured, namely gm0s1, starts up, with
consumers that I originally set up da1s1* 

My question now is, I have some idea of how GEOM works from the manpage, it
seems to say that the disk data flows through a set of geoms to/from the
kernel/disks. Will the "ghost" mirror and slices provide extra steps, and
therefore a performance slowdown, or am I totally off base with the way GEOM
works? (Which is likely since I was only introduced to it last week). That is,
is the way it is set up now on my box Ok? It works Ok, and I can reboot now
with a COMPLETE array, but is it messed up internally? I haven't done any
performance tests yet, so I don't know. 

Can anyone enlighten me on this?

Here is the text of the original message. I'm leaving it here because it
provides a lot of detail on my set up to back up my question above, although
the problem I outlined below is SOLVED and my question is above. 

Thanks,
DZ

******************************************************************************************


I've completely turned off swap for now, but I still swapoff="YES" in my
/etc/rc.conf, and I've consistently used shutdown since I learned about the
swap consequences.

I'm  running:

FreeBSD localhost 5.3-RELEASE-p4 FreeBSD 5.3-RELEASE-p4 #6: Fri Jan 14 06:42:28 CST 2005    toor at localhost:/usr/obj/usr/src/sys/MYKERNEL  i386

freshly CVSup'd on a Dell PowerEdge 2850 with this patch:

http://lists.freebsd.org/pipermail/freebsd-geom/2004-October/000338.html

with the delay set to 10 as the poster originally stated. I originally started
with the approach #2, mirror-on-a-slice, but encountered problems because on
bootup GEOM couldn't update the metadata on da0. So I went with Approach #1 in
the article, and I no longer see the problem with writing metadata, but now it
seems that it comes up DEGRADED because there is other information left over
from the original setup. When I remove da1 from the mirror the old gm0s1
mirror starts up, although I've tried every way I can think of to get rid of
it. When I remove the old mirror that is starting up, gm0s1, then
kern.geom.conftx shows da1s1 MBR's and partitions.  How do I get rid
them?  I've tried very many methods, even  going back to
scratch, removing gmirror and just booting of the /dev/da* devices, and doing
it all over again. I've even tried zeroing each disk with dd and reconnecting them,
but to no avail. I'm thinking that this is part of the problem preventing my
mirror from coming up complete after a reboot.

I've been through the freebsd-GEOM archives, but I honestly don't know what
has been folded into what I'm running now, and what other patches to try, if
that is even the problem. 

I've prepared a complete log of how it looks COMPLETE, and then after the
reboot and rebuild.

I've really wasted a good bit of time on this, I would appreciate any help,
the final goal is to have a COMPLETE mirror through reboots. 

*****************************************************************************************
Here we go, Here after a rebuild, the mirror is complete and happy.
*****************************************************************************************


gmirror list
Geom name: gm0
State: COMPLETE
Components: 2
Balance: round-robin
Slice: 4096
Flags: NONE
SyncID: 2
ID: 2505058140
Providers:
1. Name: mirror/gm0
   Mediasize: 36420074496 (34G)
   Sectorsize: 512
   Mode: r1w1e2
Consumers:
1. Name: da0
   Mediasize: 36420075008 (34G)
   Sectorsize: 512
   Mode: r1w1e3
   State: ACTIVE
   Priority: 0
   Flags: DIRTY
   SyncID: 2
   ID: 4003588011
2. Name: da1
   Mediasize: 36420075008 (34G)
   Sectorsize: 512
   Mode: r1w1e3
   State: ACTIVE
   Priority: 0
   Flags: DIRTY
   SyncID: 2
   ID: 3220781586

Geom name: gm0.sync


*****************************************************************************************
********** REBOOT  
Now we're logged in again, we see the mirror in DEGRADED mode, rebuilding,
Here's a listing:
*****************************************************************************************


root at localhost:/root# gmirror list
Geom name: gm0
State: DEGRADED
Components: 2
Balance: round-robin
Slice: 4096
Flags: NONE
SyncID: 3
ID: 2505058140
Providers:
1. Name: mirror/gm0
   Mediasize: 36420074496 (34G)
   Sectorsize: 512
   Mode: r2w1e2
Consumers:
1. Name: da0
   Mediasize: 36420075008 (34G)
   Sectorsize: 512
   Mode: r2w1e3
   State: ACTIVE
   Priority: 0
   Flags: DIRTY
   SyncID: 3
   ID: 4003588011
2. Name: da1
   Mediasize: 36420075008 (34G)
   Sectorsize: 512
   Mode: r0w1e1
   State: SYNCHRONIZING
   Priority: 0
   Flags: DIRTY, SYNCHRONIZING
   SyncID: 3
   Synchronized: 17%
   ID: 3220781586

Geom name: gm0.sync
Consumers:
1. Name: mirror/gm0
   Mediasize: 36420074496 (34G)
   Sectorsize: 512
   Mode: r1w0e0

*****************************************************************************************
*************
****** What kern.geom.conftx during the rebuild
********
*****************************************************************************************

# sysctl -b kern.geom.conftx 
0 DISK da1 36420075008 512 hd 255 sc 63
1 MIRROR mirror/gm0 36420074496 512
2 MBR mirror/gm0s1 36413282304 512 i 0 o 32256 ty 165
3 BSD mirror/gm0s1c 36413282304 512 i 2 o 0 ty 0
3 BSD mirror/gm0s1b 1073733632 512 i 1 o 35339548672 ty 1
3 BSD mirror/gm0s1a 35339540480 512 i 0 o 8192 ty 7
0 DISK da0 36420075008 512 hd 255 sc 63
1 MIRROR mirror/gm0 36420074496 512
2 MBR mirror/gm0s1 36413282304 512 i 0 o 32256 ty 165
3 BSD mirror/gm0s1c 36413282304 512 i 2 o 0 ty 0
3 BSD mirror/gm0s1b 1073733632 512 i 1 o 35339548672 ty 1
3 BSD mirror/gm0s1a 35339540480 512 i 0 o 8192 ty 7

*****************************************************************************************
*******************
*******  And here's /var/log/messages from bootup, until this point
******************
*****************************************************************************************




Jan 17 06:43:55 localhost kernel: da0 at mpt0 bus 0 target 0 lun 0
Jan 17 06:43:55 localhost kernel: da0: <SEAGATE ST336607LC DS09> Fixed Direct Access SCSI-3 device 
Jan 17 06:43:55 localhost kernel: da0: 320.000MB/s transfers (160.000MHz, offset 63, 16bit), Tagged Queueing Enabled
Jan 17 06:43:55 localhost kernel: da0: 34732MB (71132959 512 byte sectors: 255H 63S/T 4427C)
Jan 17 06:43:55 localhost kernel: da1 at mpt0 bus 0 target 1 lun 0
Jan 17 06:43:55 localhost kernel: da1: <SEAGATE ST336607LC DS09> Fixed Direct Access SCSI-3 device 
Jan 17 06:43:55 localhost kernel: da1: 320.000MB/s transfers (160.000MHz, offset 63, 16bit), Tagged Queueing Enabled
Jan 17 06:43:55 localhost kernel: da1: 34732MB (71132959 512 byte sectors: 255H 63S/T 4427C)
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Tasting da0.
Jan 17 06:43:55 localhost kernel: magic: GEOM::MIRROR
Jan 17 06:43:55 localhost kernel: version: 1
Jan 17 06:43:55 localhost kernel: name: gm0
Jan 17 06:43:55 localhost kernel: mid: 2505058140
Jan 17 06:43:55 localhost kernel: did: 4003588011
Jan 17 06:43:55 localhost kernel: all: 2
Jan 17 06:43:55 localhost kernel: syncid: 2
Jan 17 06:43:55 localhost kernel: priority: 0
Jan 17 06:43:55 localhost kernel: slice: 4096
Jan 17 06:43:55 localhost kernel: balance: round-robin
Jan 17 06:43:55 localhost kernel: mediasize: 36420074496
Jan 17 06:43:55 localhost kernel: sectorsize: 512
Jan 17 06:43:55 localhost kernel: syncoffset: 0
Jan 17 06:43:55 localhost kernel: mflags: NONE
Jan 17 06:43:55 localhost kernel: dflags: NONE
Jan 17 06:43:55 localhost kernel: hcprovider: 
Jan 17 06:43:55 localhost kernel: MD5 hash: 05b629f9060ab32a0d273902be366a37
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[1]: Creating device gm0 (id=2505058140).
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[0]: Device gm0 created (id=2505058140).
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[1]: Adding disk da0 to gm0.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Adding disk da0.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Disk da0 connected.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[1]: Disk da0 state changed from NONE to NEW (device gm0).
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[0]: Device gm0: provider da0 detected.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Tasting da1.
Jan 17 06:43:55 localhost kernel: magic: GEOM::MIRROR
Jan 17 06:43:55 localhost kernel: version: 1
Jan 17 06:43:55 localhost kernel: name: gm0
Jan 17 06:43:55 localhost kernel: mid: 2505058140
Jan 17 06:43:55 localhost kernel: did: 3220781586
Jan 17 06:43:55 localhost kernel: all: 2
Jan 17 06:43:55 localhost kernel: syncid: 2
Jan 17 06:43:55 localhost kernel: priority: 0
Jan 17 06:43:55 localhost kernel: slice: 4096
Jan 17 06:43:55 localhost kernel: balance: round-robin
Jan 17 06:43:55 localhost kernel: mediasize: 36420074496
Jan 17 06:43:55 localhost kernel: sectorsize: 512
Jan 17 06:43:55 localhost kernel: syncoffset: 0
Jan 17 06:43:55 localhost kernel: mflags: NONE
Jan 17 06:43:55 localhost kernel: dflags: NONE
Jan 17 06:43:55 localhost kernel: hcprovider: 
Jan 17 06:43:55 localhost kernel: MD5 hash: b4ab3313a1ead3ed591d0b3f4deed554
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[1]: Adding disk da1 to gm0.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Adding disk da1.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Disk da1 connected.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[1]: Disk da1 state changed from NONE to NEW (device gm0).
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[0]: Device gm0: provider da1 detected.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[1]: Device gm0 state changed from STARTING to RUNNING.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[1]: Disk da1 state changed from NEW to ACTIVE (device gm0).
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r0w1e1 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r0w-1e-1 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Metadata on da1 updated.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[0]: Device gm0: provider da1 activated.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[1]: Disk da0 state changed from NEW to ACTIVE (device gm0).
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r0w1e1 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Tasting da0s1.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Tasting da1s1.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r0w-1e-1 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Metadata on da0 updated.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[0]: Device gm0: provider da0 activated.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[0]: Device gm0: provider mirror/gm0 launched.
Jan 17 06:43:55 localhost kernel: magic: GEOM::MIRROR
Jan 17 06:43:55 localhost kernel: version: 1
Jan 17 06:43:55 localhost kernel: name: gm0s1
Jan 17 06:43:55 localhost kernel: mid: 3061091415
Jan 17 06:43:55 localhost kernel: did: 3644757297
Jan 17 06:43:55 localhost kernel: all: 2
Jan 17 06:43:55 localhost kernel: syncid: 17
Jan 17 06:43:55 localhost kernel: priority: 0
Jan 17 06:43:55 localhost kernel: slice: 4096
Jan 17 06:43:55 localhost kernel: balance: round-robin
Jan 17 06:43:55 localhost kernel: mediasize: 36413281792
Jan 17 06:43:55 localhost kernel: sectorsize: 512
Jan 17 06:43:55 localhost kernel: syncoffset: 0
Jan 17 06:43:55 localhost kernel: mflags: NONE
Jan 17 06:43:55 localhost kernel: dflags: NONE
Jan 17 06:43:55 localhost kernel: hcprovider: da1s1
Jan 17 06:43:55 localhost kernel: MD5 hash: a3fe1554ce5cb9364934713102300b18
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[1]: Creating device gm0s1 (id=3061091415).
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[0]: Device gm0s1 created (id=3061091415).
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[1]: Adding disk da1s1 to gm0s1.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Adding disk da1s1.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Disk da1s1 connected.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[1]: Disk da1s1 state changed from NONE to NEW (device gm0s1).
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[0]: Device gm0s1: provider da1s1 detected.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r1w0e0.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r1w0e1 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w0e1 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r-1w0e0.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r-1w0e-1 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r-1w0e-1 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Tasting mirror/gm0.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r1w0e0.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r1w0e1 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w0e1 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r-1w0e0.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r-1w0e-1 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r-1w0e-1 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r1w0e0.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r1w0e1 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w0e1 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r-1w0e0.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r-1w0e-1 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r-1w0e-1 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r1w0e0.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r1w0e1 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w0e1 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r-1w0e0.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r-1w0e-1 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r-1w0e-1 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Tasting da1s1a.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Tasting da1s1b.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Tasting da1s1c.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r1w0e1.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r1w0e2 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w0e2 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r-1w0e-1.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r-1w0e-2 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r-1w0e-2 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Tasting mirror/gm0s1.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r1w0e1.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r1w0e2 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w0e2 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r-1w0e-1.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r-1w0e-2 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r-1w0e-2 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r1w0e1.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r1w0e2 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w0e2 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r-1w0e-1.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r-1w0e-2 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r-1w0e-2 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r1w0e1.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r1w0e2 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w0e2 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r-1w0e-1.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r-1w0e-2 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r-1w0e-2 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r1w0e1.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r1w0e2 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w0e2 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r-1w0e-1.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r-1w0e-2 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r-1w0e-2 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r1w0e2.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r1w0e3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w0e3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r-1w0e-2.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r-1w0e-3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r-1w0e-3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Tasting mirror/gm0s1a.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r1w0e2.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r1w0e3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w0e3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r-1w0e-2.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r-1w0e-3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r-1w0e-3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r1w0e2.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r1w0e3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w0e3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r-1w0e-2.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r-1w0e-3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r-1w0e-3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r1w0e2.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r1w0e3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w0e3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r-1w0e-2.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r-1w0e-3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r-1w0e-3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r1w0e2.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r1w0e3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w0e3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r-1w0e-2.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r-1w0e-3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r-1w0e-3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Tasting mirror/gm0s1b.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r1w0e2.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r1w0e3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w0e3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r-1w0e-2.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r-1w0e-3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r-1w0e-3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r1w0e2.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r1w0e3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w0e3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r-1w0e-2.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r-1w0e-3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r-1w0e-3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r1w0e2.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r1w0e3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w0e3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r-1w0e-2.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r-1w0e-3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r-1w0e-3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r1w0e2.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r1w0e3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w0e3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r-1w0e-2.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r-1w0e-3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r-1w0e-3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Tasting mirror/gm0s1c.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r1w0e2.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r1w0e3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w0e3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r-1w0e-2.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r-1w0e-3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r-1w0e-3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r1w0e2.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r1w0e3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w0e3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r-1w0e-2.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r-1w0e-3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r-1w0e-3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r1w0e2.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r1w0e3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w0e3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r-1w0e-2.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r-1w0e-3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r-1w0e-3 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[0]: Force device gm0s1 start due to timeout.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[1]: Device gm0s1 state changed from STARTING to RUNNING.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[1]: Disk da1s1 state changed from NEW to ACTIVE (device gm0s1).
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r0w1e1 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[1]: Disk da1 state changed from ACTIVE to DISCONNECTED (device gm0).
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[0]: Device gm0: provider da1 disconnected.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Disk da1 disconnected.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Consumer da1 destroyed.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[1]: Device gm0: syncid bumped to 3.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r0w1e1 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r0w-1e-1 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Metadata on da1s1 updated.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[0]: Device gm0s1: provider da1s1 activated.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[0]: Device gm0s1: provider mirror/gm0s1 launched.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r0w-1e-1 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Metadata on da0 updated.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Tasting da1.
Jan 17 06:43:55 localhost kernel: magic: GEOM::MIRROR
Jan 17 06:43:55 localhost kernel: version: 1
Jan 17 06:43:55 localhost kernel: name: gm0
Jan 17 06:43:55 localhost kernel: mid: 2505058140
Jan 17 06:43:55 localhost kernel: did: 3220781586
Jan 17 06:43:55 localhost kernel: all: 2
Jan 17 06:43:55 localhost kernel: syncid: 2
Jan 17 06:43:55 localhost kernel: priority: 0
Jan 17 06:43:55 localhost kernel: slice: 4096
Jan 17 06:43:55 localhost kernel: balance: round-robin
Jan 17 06:43:55 localhost kernel: mediasize: 36420074496
Jan 17 06:43:55 localhost kernel: sectorsize: 512
Jan 17 06:43:55 localhost kernel: syncoffset: 0
Jan 17 06:43:55 localhost kernel: mflags: NONE
Jan 17 06:43:55 localhost kernel: dflags: NONE
Jan 17 06:43:55 localhost kernel: hcprovider: 
Jan 17 06:43:55 localhost kernel: MD5 hash: 7a64c716e9497f74a61b8dc8be0dad3f
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[1]: Adding disk da1 to gm0.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Adding disk da1.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Disk da1 connected.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[1]: Disk da1 state changed from NONE to NEW (device gm0).
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[0]: Device gm0: provider da1 detected.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[1]: Disk da1 state changed from NEW to SYNCHRONIZING (device gm0).
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[0]: Device gm0: rebuilding provider da1.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1 r0w1e1 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r1w0e0.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w0e1 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Metadata on da1 updated.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0s1: r1w0e0.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r1w0e1 = 6
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Tasting mirror/gm0s1.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0s1: r1w0e0.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r1w0e1 = 6
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0s1: r1w0e0.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r1w0e1 = 6
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0s1: r1w0e0.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r1w0e1 = 6
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[1]: Disk da1s1 state changed from ACTIVE to DISCONNECTED (device gm0s1).
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[0]: Device gm0s1: provider da1s1 disconnected.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Disk da1s1 disconnected.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Consumer da1s1 destroyed.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: No I/O requests for gm0s1, it can be destroyed.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[0]: Device gm0s1: provider mirror/gm0s1 destroyed.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[0]: Device gm0s1 destroyed.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Tasting da0s1.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Tasting da0s1a.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Tasting da0s1b.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Tasting da0s1c.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r1w1e2.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w1e2 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[1]: Disk da0 (device gm0) marked as dirty.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Metadata on da0 updated.
Jan 17 06:43:55 localhost kernel: Mounted root from ufs:/dev/mirror/gm0s1a.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Metadata on da1 updated.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r1w0e0.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w0e0 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r-1w0e0.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r-1w0e0 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r1w0e0.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w0e0 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r0w1e0.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r0w1e0 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r-1w0e0.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r-1w0e0 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r0w-1e0.
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Access da0 r0w-1e0 = 0
Jan 17 06:43:55 localhost kernel: GEOM_MIRROR[2]: Metadata on da1 updated.
Jan 17 06:45:09 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r1w0e0.
Jan 17 06:45:09 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w0e0 = 0
Jan 17 06:45:09 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r-1w0e0.
Jan 17 06:45:09 localhost kernel: GEOM_MIRROR[2]: Access da0 r-1w0e0 = 0
Jan 17 06:45:10 localhost kernel: GEOM_MIRROR[2]: Metadata on da1 updated.
Jan 17 06:53:07 localhost last message repeated 26 times



*****************************************************************************************
********************
***** Now I disconnect da1 from gm0 I originally did this because I wanted to
just zero /dev/da1 with dd and start over (or so I thought). But when I ran a
list, I was surprised to see gm0s1 starting up:

************************
*****************************************************************************************
root at localhost:/root# gmirror deactivate gm0 da1
root at localhost:/root# gmirror list
Geom name: gm0
State: DEGRADED
Components: 2
Balance: round-robin
Slice: 4096
Flags: NONE
SyncID: 4
ID: 2505058140
Providers:
1. Name: mirror/gm0
   Mediasize: 36420074496 (34G)
   Sectorsize: 512
   Mode: r1w1e2
Consumers:
1. Name: da0
   Mediasize: 36420075008 (34G)
   Sectorsize: 512
   Mode: r1w1e3
   State: ACTIVE
   Priority: 0
   Flags: DIRTY
   SyncID: 4
   ID: 4003588011

Geom name: gm0.sync

Geom name: gm0s1
State: DEGRADED
Components: 2
Balance: round-robin
Slice: 4096
Flags: NONE
SyncID: 17
ID: 3061091415
Providers:
1. Name: mirror/gm0s1
   Mediasize: 36413281792 (34G)
   Sectorsize: 512
   Mode: r0w0e0
Consumers:
1. Name: da1s1
   Mediasize: 36413282304 (34G)
   Sectorsize: 512
   Mode: r0w0e0
   State: ACTIVE
   Priority: 0
   Flags: HARDCODED
   SyncID: 17
   ID: 3644757297

Geom name: gm0s1.sync

*****************************************************************************************
**********************************
**** Now kern.geom.conftx looked like this
*****************************************************************************************
*************************

# sysctl -b kern.geom.conftx
0 DISK da1 36420075008 512 hd 255 sc 63
1 MBR da1s1 36413282304 512 i 0 o 32256 ty 165
2 BSD da1s1c 36413282304 512 i 2 o 0 ty 0
2 BSD da1s1b 1073733632 512 i 1 o 35339548672 ty 1
2 BSD da1s1a 35339540480 512 i 0 o 8192 ty 7
2 MIRROR mirror/gm0s1 36413281792 512
3 BSD mirror/gm0s1c 36413282304 512 i 2 o 0 ty 0
3 BSD mirror/gm0s1b 1073733632 512 i 1 o 35339548672 ty 1
3 BSD mirror/gm0s1a 35339540480 512 i 0 o 8192 ty 7
0 DISK da0 36420075008 512 hd 255 sc 63
1 MIRROR mirror/gm0 36420074496 512
2 MBR mirror/gm0s1 36413282304 512 i 0 o 32256 ty 165
3 BSD mirror/gm0s1c 36413282304 512 i 2 o 0 ty 0
3 BSD mirror/gm0s1b 1073733632 512 i 1 o 35339548672 ty 1
3 BSD mirror/gm0s1a 35339540480 512 i 0 o 8192 ty 7

*****************************************************************************************
***********************************
**** /var/log/messages for this event:
**********************************
*****************************************************************************************

Jan 17 06:53:07 localhost kernel: GEOM_MIRROR[1]: Disk da1 state changed from SYNCHRONIZING to DISCONNECTED (device gm0).
Jan 17 06:53:07 localhost kernel: GEOM_MIRROR[0]: Device gm0: provider da1 disconnected.
Jan 17 06:53:07 localhost kernel: GEOM_MIRROR[0]: Device gm0: rebuilding provider da1 stopped.
Jan 17 06:53:07 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r-1w0e0.
Jan 17 06:53:07 localhost kernel: GEOM_MIRROR[2]: Access da0 r-1w0e0 = 0
Jan 17 06:53:07 localhost kernel: GEOM_MIRROR[2]: Consumer mirror/gm0 destroyed.
Jan 17 06:53:07 localhost kernel: GEOM_MIRROR[2]: Access da1 r0w-1e-1 = 0
Jan 17 06:53:07 localhost kernel: GEOM_MIRROR[2]: Disk da1 disconnected.
Jan 17 06:53:07 localhost kernel: GEOM_MIRROR[2]: I/O requests for da1 in queue, can't destroy it now.
Jan 17 06:53:07 localhost kernel: GEOM_MIRROR[2]: Consumer da1 destroyed.
Jan 17 06:53:07 localhost kernel: GEOM_MIRROR[2]: Tasting da1.
Jan 17 06:53:07 localhost kernel: GEOM_MIRROR[0]: Device gm0: provider da1 marked as inactive, skipping.
Jan 17 06:53:07 localhost kernel: GEOM_MIRROR[2]: Tasting da1s1.
Jan 17 06:53:07 localhost kernel: magic: GEOM::MIRROR
Jan 17 06:53:07 localhost kernel: version: 1
Jan 17 06:53:07 localhost kernel: name: gm0s1
Jan 17 06:53:07 localhost kernel: mid: 3061091415
Jan 17 06:53:07 localhost kernel: did: 3644757297
Jan 17 06:53:07 localhost kernel: all: 2
Jan 17 06:53:07 localhost kernel: syncid: 17
Jan 17 06:53:07 localhost kernel: priority: 0
Jan 17 06:53:07 localhost kernel: slice: 4096
Jan 17 06:53:07 localhost kernel: balance: round-robin
Jan 17 06:53:07 localhost kernel: mediasize: 36413281792
Jan 17 06:53:07 localhost kernel: sectorsize: 512
Jan 17 06:53:07 localhost kernel: syncoffset: 0
Jan 17 06:53:07 localhost kernel: mflags: NONE
Jan 17 06:53:07 localhost kernel: dflags: NONE
Jan 17 06:53:07 localhost kernel: hcprovider: da1s1
Jan 17 06:53:07 localhost kernel: MD5 hash: 67a736074371d7996a699af47c6fb79c
Jan 17 06:53:07 localhost kernel: GEOM_MIRROR[1]: Creating device gm0s1 (id=3061091415).
Jan 17 06:53:07 localhost kernel: GEOM_MIRROR[0]: Device gm0s1 created (id=3061091415).
Jan 17 06:53:07 localhost kernel: GEOM_MIRROR[1]: Adding disk da1s1 to gm0s1.
Jan 17 06:53:07 localhost kernel: GEOM_MIRROR[2]: Adding disk da1s1.
Jan 17 06:53:07 localhost kernel: GEOM_MIRROR[2]: Disk da1s1 connected.
Jan 17 06:53:07 localhost kernel: GEOM_MIRROR[1]: Disk da1s1 state changed from NONE to NEW (device gm0s1).
Jan 17 06:53:07 localhost kernel: GEOM_MIRROR[0]: Device gm0s1: provider da1s1 detected.
Jan 17 06:53:07 localhost kernel: GEOM_MIRROR[2]: Tasting da1s1a.
Jan 17 06:53:07 localhost kernel: GEOM_MIRROR[1]: Device gm0: syncid bumped to 4.
Jan 17 06:53:07 localhost kernel: GEOM_MIRROR[2]: Metadata on da0 updated.
Jan 17 06:53:07 localhost kernel: GEOM_MIRROR[2]: Tasting da1s1b.
Jan 17 06:53:07 localhost kernel: GEOM_MIRROR[2]: Tasting da1s1c.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[0]: Force device gm0s1 start due to timeout.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[1]: Device gm0s1 state changed from STARTING to RUNNING.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[1]: Disk da1s1 state changed from NEW to ACTIVE (device gm0s1).
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r0w1e1 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r0w-1e-1 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Metadata on da1s1 updated.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[0]: Device gm0s1: provider da1s1 activated.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[0]: Device gm0s1: provider mirror/gm0s1 launched.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Tasting da1.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[0]: Device gm0: provider da1 marked as inactive, skipping.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0s1: r1w0e0.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r1w0e1 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0s1: r-1w0e0.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r-1w0e-1 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Tasting mirror/gm0s1.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0s1: r1w0e0.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r1w0e1 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0s1: r-1w0e0.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r-1w0e-1 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0s1: r1w0e0.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r1w0e1 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0s1: r-1w0e0.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r-1w0e-1 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0s1: r1w0e0.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r1w0e1 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0s1: r-1w0e0.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r-1w0e-1 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Tasting da1s1a.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Tasting da1s1b.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Tasting da1s1c.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0s1: r1w0e1.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r1w0e2 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0s1: r-1w0e-1.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r-1w0e-2 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Tasting mirror/gm0s1a.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0s1: r1w0e1.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r1w0e2 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0s1: r-1w0e-1.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r-1w0e-2 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0s1: r1w0e1.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r1w0e2 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0s1: r-1w0e-1.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r-1w0e-2 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0s1: r1w0e1.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r1w0e2 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0s1: r-1w0e-1.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r-1w0e-2 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0s1: r1w0e1.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r1w0e2 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0s1: r-1w0e-1.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r-1w0e-2 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Tasting mirror/gm0s1b.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0s1: r1w0e1.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r1w0e2 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0s1: r-1w0e-1.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r-1w0e-2 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0s1: r1w0e1.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r1w0e2 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0s1: r-1w0e-1.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r-1w0e-2 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0s1: r1w0e1.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r1w0e2 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0s1: r-1w0e-1.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r-1w0e-2 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0s1: r1w0e1.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r1w0e2 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0s1: r-1w0e-1.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r-1w0e-2 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Tasting mirror/gm0s1c.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0s1: r1w0e1.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r1w0e2 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0s1: r-1w0e-1.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r-1w0e-2 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0s1: r1w0e1.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r1w0e2 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0s1: r-1w0e-1.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r-1w0e-2 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0s1: r1w0e1.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r1w0e2 = 0
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0s1: r-1w0e-1.
Jan 17 06:53:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r-1w0e-2 = 0



*****************************************************************************************
*************************
********* forgetting on gm0s1 so I can remove it
**************************
*****************************************************************************************

root at localhost:/root# gmirror  forget gm0s1
root at localhost:/root# gmirror list
Geom name: gm0
State: DEGRADED
Components: 2
Balance: round-robin
Slice: 4096
Flags: NONE
SyncID: 4
ID: 2505058140
Providers:
1. Name: mirror/gm0
   Mediasize: 36420074496 (34G)
   Sectorsize: 512
   Mode: r1w1e2
Consumers:
1. Name: da0
   Mediasize: 36420075008 (34G)
   Sectorsize: 512
   Mode: r1w1e3
   State: ACTIVE
   Priority: 0
   Flags: DIRTY
   SyncID: 4
   ID: 4003588011

Geom name: gm0.sync

Geom name: gm0s1
State: COMPLETE
Components: 1
Balance: round-robin
Slice: 4096
Flags: NONE
SyncID: 17
ID: 3061091415
Providers:
1. Name: mirror/gm0s1
   Mediasize: 36413281792 (34G)
   Sectorsize: 512
   Mode: r0w0e0
Consumers:
1. Name: da1s1
   Mediasize: 36413282304 (34G)
   Sectorsize: 512
   Mode: r0w0e0
   State: ACTIVE
   Priority: 0
   Flags: HARDCODED
   SyncID: 17
   ID: 3644757297

Geom name: gm0s1.sync

*****************************************************************************************
***********************
********** So, now I want to remove gm0s1, but I can't because it always
********** comes back, I need to manually destroy it each time and then add
da1 to gm0
*****************************************************************************************
**************************

root at localhost:/root# gmirror remove gm0s1 da1s1


*************************
0 DISK da1 36420075008 512 hd 255 sc 63
1 MBR da1s1 36413282304 512 i 0 o 32256 ty 165
2 MIRROR mirror/gm0s1 36413281792 512
3 BSD mirror/gm0s1c 36413282304 512 i 2 o 0 ty 0
3 BSD mirror/gm0s1b 1073733632 512 i 1 o 35339548672 ty 1
3 BSD mirror/gm0s1a 35339540480 512 i 0 o 8192 ty 7
0 DISK da0 36420075008 512 hd 255 sc 63
1 MIRROR mirror/gm0 36420074496 512
2 MBR mirror/gm0s1 36413282304 512 i 0 o 32256 ty 165
3 BSD mirror/gm0s1c 36413282304 512 i 2 o 0 ty 0
3 BSD mirror/gm0s1b 1073733632 512 i 1 o 35339548672 ty 1
3 BSD mirror/gm0s1a 35339540480 512 i 0 o 8192 ty 7


*****************************************************************************************
**********************
********** Removing da1s1 from gm0s1 reveals the da1s1 MBR and the partitions
********** on da1  Huh?
*************************
*****************************************************************************************


root at localhost:/root# gmirror remove gm0s1 da1s1
root at localhost:/root# gmirror list
Geom name: gm0
State: DEGRADED
Components: 2
Balance: round-robin
Slice: 4096
Flags: NONE
SyncID: 4
ID: 2505058140
Providers:
1. Name: mirror/gm0
   Mediasize: 36420074496 (34G)
   Sectorsize: 512
   Mode: r1w1e2
Consumers:
1. Name: da0
   Mediasize: 36420075008 (34G)
   Sectorsize: 512
   Mode: r1w1e3
   State: ACTIVE
   Priority: 0
   Flags: DIRTY
   SyncID: 4
   ID: 4003588011

Geom name: gm0.sync

0 DISK da1 36420075008 512 hd 255 sc 63
1 MBR da1s1 36413282304 512 i 0 o 32256 ty 165
2 BSD da1s1c 36413282304 512 i 2 o 0 ty 0
2 BSD da1s1b 1073733632 512 i 1 o 35339548672 ty 1
2 BSD da1s1a 35339540480 512 i 0 o 8192 ty 7
0 DISK da0 36420075008 512 hd 255 sc 63
1 MIRROR mirror/gm0 36420074496 512
2 MBR mirror/gm0s1 36413282304 512 i 0 o 32256 ty 165
3 BSD mirror/gm0s1c 36413282304 512 i 2 o 0 ty 0
3 BSD mirror/gm0s1b 1073733632 512 i 1 o 35339548672 ty 1
3 BSD mirror/gm0s1a 35339540480 512 i 0 o 8192 ty 7


*****************************************************************************************
So I forget on gm0 so I can work on it,

insert da1 and I'm back in business, kern.geom.conftx looks ok now?
*****************************************************************************************



root at localhost:/root# gmirror list
Geom name: gm0
State: COMPLETE
Components: 1
Balance: round-robin
Slice: 4096
Flags: NONE
SyncID: 4
ID: 2505058140
Providers:
1. Name: mirror/gm0
   Mediasize: 36420074496 (34G)
   Sectorsize: 512
   Mode: r1w1e2
Consumers:
1. Name: da0
   Mediasize: 36420075008 (34G)
   Sectorsize: 512
   Mode: r1w1e3
   State: ACTIVE
   Priority: 0
   Flags: DIRTY
   SyncID: 4
   ID: 4003588011

Geom name: gm0.sync

root at localhost:/root# gmirror insert gm0 da1
root at localhost:/root# gmirror list
Geom name: gm0
State: DEGRADED
Components: 2
Balance: round-robin
Slice: 4096
Flags: NONE
SyncID: 4
ID: 2505058140
Providers:
1. Name: mirror/gm0
   Mediasize: 36420074496 (34G)
   Sectorsize: 512
   Mode: r2w1e2
Consumers:
1. Name: da0
   Mediasize: 36420075008 (34G)
   Sectorsize: 512
   Mode: r2w1e3
   State: ACTIVE
   Priority: 0
   Flags: DIRTY
   SyncID: 4
   ID: 4003588011
2. Name: da1
   Mediasize: 36420075008 (34G)
   Sectorsize: 512
   Mode: r0w1e1
   State: SYNCHRONIZING
   Priority: 0
   Flags: DIRTY, SYNCHRONIZING
   SyncID: 4
   Synchronized: 0%
   ID: 3782877908

Geom name: gm0.sync
Consumers:
1. Name: mirror/gm0
   Mediasize: 36420074496 (34G)
   Sectorsize: 512
   Mode: r1w0e0

0 DISK da1 36420075008 512 hd 255 sc 63
1 MIRROR mirror/gm0 36420074496 512
2 MBR mirror/gm0s1 36413282304 512 i 0 o 32256 ty 165
3 BSD mirror/gm0s1c 36413282304 512 i 2 o 0 ty 0
3 BSD mirror/gm0s1b 1073733632 512 i 1 o 35339548672 ty 1
3 BSD mirror/gm0s1a 35339540480 512 i 0 o 8192 ty 7
0 DISK da0 36420075008 512 hd 255 sc 63
1 MIRROR mirror/gm0 36420074496 512
2 MBR mirror/gm0s1 36413282304 512 i 0 o 32256 ty 165
3 BSD mirror/gm0s1c 36413282304 512 i 2 o 0 ty 0
3 BSD mirror/gm0s1b 1073733632 512 i 1 o 35339548672 ty 1
3 BSD mirror/gm0s1a 35339540480 512 i 0 o 8192 ty 7



*****************************************************************************************
And the rest of the log entries for all of this;
*****************************************************************************************



Jan 17 06:54:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r0w1e1 = 0
Jan 17 06:54:11 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r0w-1e-1 = 0
Jan 17 06:54:11 localhost kernel: GEOM_MIRROR[2]: Metadata on da1s1 updated.
Jan 17 06:54:11 localhost kernel: GEOM_MIRROR[2]: Tasting da1.
Jan 17 06:54:11 localhost kernel: GEOM_MIRROR[0]: Device gm0: provider da1 marked as inactive, skipping.
Jan 17 06:55:18 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r0w1e1 = 0
Jan 17 06:55:18 localhost kernel: GEOM_MIRROR[2]: Access da1s1 r0w-1e-1 = 0
Jan 17 06:55:18 localhost kernel: GEOM_MIRROR[2]: Metadata on da1s1 cleared.
Jan 17 06:55:18 localhost kernel: GEOM_MIRROR[1]: Disk da1s1 state changed from ACTIVE to DESTROY (device gm0s1).
Jan 17 06:55:18 localhost kernel: GEOM_MIRROR[0]: Device gm0s1: provider da1s1 destroyed.
Jan 17 06:55:18 localhost kernel: GEOM_MIRROR[2]: Disk da1s1 disconnected.
Jan 17 06:55:18 localhost kernel: GEOM_MIRROR[2]: Consumer da1s1 destroyed.
Jan 17 06:55:18 localhost kernel: GEOM_MIRROR[2]: No I/O requests for gm0s1, it can be destroyed.
Jan 17 06:55:18 localhost kernel: GEOM_MIRROR[0]: Device gm0s1: provider mirror/gm0s1 destroyed.
Jan 17 06:55:18 localhost kernel: GEOM_MIRROR[0]: Device gm0s1 destroyed.
Jan 17 06:55:18 localhost kernel: GEOM_MIRROR[2]: Tasting da1.
Jan 17 06:55:18 localhost kernel: GEOM_MIRROR[0]: Device gm0: provider da1 marked as inactive, skipping.
Jan 17 06:55:18 localhost kernel: GEOM_MIRROR[2]: Tasting da1s1.
Jan 17 06:55:18 localhost kernel: GEOM_MIRROR[2]: Tasting da1s1a.
Jan 17 06:55:19 localhost kernel: GEOM_MIRROR[2]: Tasting da1s1b.
Jan 17 06:55:19 localhost kernel: GEOM_MIRROR[2]: Tasting da1s1c.
Jan 17 06:56:05 localhost kernel: GEOM_MIRROR[2]: Metadata on da0 updated.
Jan 17 06:56:22 localhost kernel: GEOM_MIRROR[2]: Metadata on da0 updated.
Jan 17 06:56:22 localhost kernel: GEOM_MIRROR[2]: Tasting da1.
Jan 17 06:56:22 localhost kernel: magic: GEOM::MIRROR
Jan 17 06:56:22 localhost kernel: version: 1
Jan 17 06:56:22 localhost kernel: name: gm0
Jan 17 06:56:22 localhost kernel: mid: 2505058140
Jan 17 06:56:22 localhost kernel: did: 3782877908
Jan 17 06:56:22 localhost kernel: all: 2
Jan 17 06:56:22 localhost kernel: syncid: 0
Jan 17 06:56:22 localhost kernel: priority: 0
Jan 17 06:56:22 localhost kernel: slice: 4096
Jan 17 06:56:22 localhost kernel: balance: round-robin
Jan 17 06:56:22 localhost kernel: mediasize: 36420074496
Jan 17 06:56:22 localhost kernel: sectorsize: 512
Jan 17 06:56:22 localhost kernel: syncoffset: 0
Jan 17 06:56:22 localhost kernel: mflags: NONE
Jan 17 06:56:22 localhost kernel: dflags: NONE
Jan 17 06:56:22 localhost kernel: hcprovider: 
Jan 17 06:56:22 localhost kernel: MD5 hash: c8505c5e0fcf859952eaab3439f1b7a5
Jan 17 06:56:22 localhost kernel: GEOM_MIRROR[1]: Adding disk da1 to gm0.
Jan 17 06:56:22 localhost kernel: GEOM_MIRROR[2]: Adding disk da1.
Jan 17 06:56:22 localhost kernel: GEOM_MIRROR[2]: Disk da1 connected.
Jan 17 06:56:22 localhost kernel: GEOM_MIRROR[1]: Disk da1 state changed from NONE to NEW (device gm0).
Jan 17 06:56:22 localhost kernel: GEOM_MIRROR[0]: Device gm0: provider da1 detected.
Jan 17 06:56:22 localhost kernel: GEOM_MIRROR[1]: Disk da1 state changed from NEW to SYNCHRONIZING (device gm0).
Jan 17 06:56:22 localhost kernel: GEOM_MIRROR[0]: Device gm0: rebuilding provider da1.
Jan 17 06:56:22 localhost kernel: GEOM_MIRROR[2]: Access da1 r0w1e1 = 0
Jan 17 06:56:22 localhost kernel: GEOM_MIRROR[2]: Access request for mirror/gm0: r1w0e0.
Jan 17 06:56:22 localhost kernel: GEOM_MIRROR[2]: Access da0 r1w0e0 = 0
Jan 17 06:56:22 localhost kernel: GEOM_MIRROR[2]: Metadata on da1 updated.
Jan 17 06:56:51 localhost last message repeated 28 times
Jan 17 06:56:51 localhost ntpd[452]: kernel time sync enabled 2001
Jan 17 06:56:52 localhost kernel: GEOM_MIRROR[2]: Metadata on da1 updated.
Jan 17 06:57:23 localhost last message repeated 30 times

*****************************************************************************************
Just in case, here is the contents of /dev, filtered for da*:
*****************************************************************************************

crw-r-----  1 root  operator    4,  12 Jan 17 06:43 /dev/da0
crw-r-----  1 root  operator    4,  13 Jan 17 06:43 /dev/da1




I've been banging my head on this one for awhile. Any help would be appreciated. 

Thanks,
DZ





More information about the freebsd-geom mailing list