problems with GEOM_MIRROR
Krzysztof Ciepłucha
kris at home.pl
Fri Sep 3 05:11:34 PDT 2004
Hi,
I am testing mirroring using GEOM_MIRROR and have found several a little bit
strange behaviours.
But first my configuration (in short):
The server is using adaptec scsi controller with two scsi disks:
ibm235# camcontrol devlist
<IBM-ESXS ST336605LC !# B245> at scbus1 target 0 lun 0 (pass1,da0)
<IBM-ESXS ST336605LC !# B245> at scbus1 target 1 lun 0 (pass2,da1)
The kernel is compiled from fresh cvsup-ed 5.3BETA2 sources with statically
added GEOM_LABEL and GEOM_MIRROR.
On top of this disks I have put a labels using GEOM_LABEL: label/disk0 on
da0 and label/disk1 on da1.
Then I have made a mirror (GEOM_MIRROR) using those labels, with hardcoded
providers: mirror/mirror0 using label/disk0 and label/disk1.
On top of the mirror I have created a standard bsdlabel (with boot loader),
set up filesystems and moved the whole system from the third disk using dump
and restore.
ibm235# bsdlabel /dev/mirror/mirror0
# /dev/mirror/mirror0:
8 partitions:
# size offset fstype [fsize bsize bps/cpg]
a: 524288 0 4.2BSD 2048 16384 32776
b: 5191376 524288 swap
c: 71096638 0 unused 0 0 # "raw" part, don't
edit
d: 524288 5715664 4.2BSD 2048 16384 32776
e: 524288 6239952 4.2BSD 2048 16384 32776
f: 64332398 6764240 4.2BSD 2048 16384 28552
So finally I have running system booting and running from mirror/mirror1:
ibm235# cat /etc/fstab
# Device Mountpoint FStype Options Dump
Pass#
/dev/mirror/mirror0b none swap sw 0
0
/dev/mirror/mirror0a / ufs rw 1
1
/dev/mirror/mirror0e /tmp ufs rw 2
2
/dev/mirror/mirror0f /usr ufs rw 2
2
/dev/mirror/mirror0d /var ufs rw 2
2
/dev/acd0 /cdrom cd9660 ro,noauto 0 0
The mirror is fully synchronized (at least I think it is):
ibm235# geom mirror list
Geom name: mirror0
Components: 2
Balance: split
Slice: 4096
Flags: NONE
SyncID: 3
ID: 993051015
Providers:
1. Name: mirror/mirror0
Mediasize: 36401478656 (34G)
Sectorsize: 512
Mode: r5w5e1
Consumers:
1. Name: label/disk0
Mediasize: 36401479168 (34G)
Sectorsize: 512
Mode: r5w5e2
State: ACTIVE
Priority: 0
Flags: DIRTY, HARDCODED
SyncID: 3
ID: 2288148496
2. Name: label/disk1
Mediasize: 36401479168 (34G)
Sectorsize: 512
Mode: r5w5e2
State: ACTIVE
Priority: 0
Flags: DIRTY, HARDCODED
SyncID: 3
ID: 3740254788
Geom name: mirror0.sync
Finally the list of problems:
1. After server restart the mirror is being rebuilding form the beginning,
even if before restart it was fully synchronized.
Here is a fragment from dmesg:
<skipped>
Sep 3 07:30:47 ibm235 kernel: SMP: AP CPU #1 Launched!
Sep 3 07:30:47 ibm235 kernel: GEOM_LABEL: Label for provider da0 is
label/disk1.
Sep 3 07:30:47 ibm235 kernel: GEOM_LABEL: Label for provider da1 is
label/disk0.
Sep 3 07:30:47 ibm235 kernel: GEOM_MIRROR: Device mirror0 created
(id=993051015).
Sep 3 07:30:47 ibm235 kernel: GEOM_MIRROR: Device mirror0: provider
label/disk1 detected.
Sep 3 07:30:47 ibm235 kernel: GEOM_MIRROR: Device mirror0: provider
label/disk0 detected.
Sep 3 07:30:47 ibm235 kernel: GEOM_MIRROR: Device mirror0: provider
label/disk0 activated.
Sep 3 07:30:47 ibm235 kernel: GEOM_MIRROR: Device mirror0: provider
mirror/mirror0 launched.
Sep 3 07:30:47 ibm235 kernel: GEOM_MIRROR: Device mirror0: rebuilding
provider label/disk1.
Sep 3 07:30:47 ibm235 kernel: Mounting root from ufs:/dev/mirror/mirror0a
Sep 3 07:30:47 ibm235 kernel: GEOM_MIRROR: Device mirror0: provider
label/disk1 disconnected.
Sep 3 07:30:47 ibm235 kernel: GEOM_MIRROR: Device mirror0: rebuilding
provider label/disk1 stopped.
Sep 3 07:30:47 ibm235 kernel: GEOM_MIRROR: Device mirror0: provider
label/disk1 detected.
Sep 3 07:30:47 ibm235 kernel: GEOM_MIRROR: Device mirror0: rebuilding
provider label/disk1.
Sep 3 08:17:42 ibm235 kernel: GEOM_MIRROR: Device mirror0: rebuilding
provider label/disk1 finished.
Sep 3 08:17:42 ibm235 kernel: GEOM_MIRROR: Device mirror0: provider
label/disk1 activated.
2. sysctl -b kern.geom.conftxt shows BSD labels like label/disk0a ... disk0f
which shouldn't exist, should they?
ibm235# sysctl -b kern.geom.conftxt
0 DISK da1 36401479680 512 hd 255 sc 63
1 LABEL label/disk1 36401479168 512 i 0 o 0
2 MIRROR mirror/mirror0 36401478656 512
3 BSD mirror/mirror0f 32938187776 512 i 5 o 3463290880 ty 7
3 BSD mirror/mirror0e 268435456 512 i 4 o 3194855424 ty 7
3 BSD mirror/mirror0d 268435456 512 i 3 o 2926419968 ty 7
3 BSD mirror/mirror0c 36401478656 512 i 2 o 0 ty 0
3 BSD mirror/mirror0b 2657984512 512 i 1 o 268435456 ty 1
3 BSD mirror/mirror0a 268435456 512 i 0 o 0 ty 7
0 DISK da0 36401479680 512 hd 255 sc 63
1 LABEL label/disk0 36401479168 512 i 0 o 0
2 BSD label/disk0f 32938187776 512 i 5 o 3463290880 ty 7
2 BSD label/disk0e 268435456 512 i 4 o 3194855424 ty 7
2 BSD label/disk0d 268435456 512 i 3 o 2926419968 ty 7
2 BSD label/disk0c 36401478656 512 i 2 o 0 ty 0
2 BSD label/disk0b 2657984512 512 i 1 o 268435456 ty 1
2 BSD label/disk0a 268435456 512 i 0 o 0 ty 7
2 MIRROR mirror/mirror0 36401478656 512
3 BSD mirror/mirror0f 32938187776 512 i 5 o 3463290880 ty 7
3 BSD mirror/mirror0e 268435456 512 i 4 o 3194855424 ty 7
3 BSD mirror/mirror0d 268435456 512 i 3 o 2926419968 ty 7
3 BSD mirror/mirror0c 36401478656 512 i 2 o 0 ty 0
3 BSD mirror/mirror0b 2657984512 512 i 1 o 268435456 ty 1
3 BSD mirror/mirror0a 268435456 512 i 0 o 0 ty 7
Does anybody have working configuration like this?
Is there a bug in GEOM implementation, or it is a problem with my
configuration.
--
kris-at-home-dot-pl
More information about the freebsd-geom
mailing list