GEOM's RAID level support

Adrian Wontroba aw1 at stade.co.uk
Fri Oct 7 18:35:26 PDT 2005


On Thu, Oct 06, 2005 at 01:27:00PM -0700, Danny Howard wrote:

> Well, I have RAID1 systems ...

I've just reworked one of my home systems, doubling its disk capacity.

I started with a pretty standard gmirrored two disk system, added
another Promise controller and another pair of (nearly) identical disks,
setting up the new disks as:

Mirrored root (gm0).
Single plex mirror of swap (gm1) or Single plex mirror of /var and /usr (gm2)
Rest of the space striped (gs0), underneath a single plex mirror of /home (gm3)

When I'd got the new mirrors working, I manually amended /etc/fstab etc,
booted from them and setup the other stripes/ plexes on the original
disks, ending up with four plexes for /root and two for everything
else. With the ATA write cache enabled it flies. With the write cache
disabled it crawls. Must buy a UPS.

The end result looks like:

bash-2.05b# gmirror status
      Name    Status  Components
mirror/gm0  COMPLETE  ad4a
                      ad6a
                      ad8a
                      ad10a
mirror/gm1  COMPLETE  ad4d
                      ad8d
mirror/gm2  COMPLETE  ad6d
                      ad10d
mirror/gm3  COMPLETE  stripe/gs1
                      stripe/gs0
bash-2.05b# gstripe status
      Name  Status  Components
stripe/gs1      UP  ad4e
                    ad6e
stripe/gs0      UP  ad8e
                    ad10e
bash-2.05b# df
Filesystem       1K-blocks     Used     Avail Capacity  Mounted on
/dev/mirror/gm0    1012972    56370    875566     6%    /
devfs                    1        1         0   100%    /dev
/dev/mirror/gm2d   1012974    18558    913380     2%    /var
/dev/mirror/gm2e   3045004  1398126   1403278    50%    /usr
/dev/mirror/gm3  292588412 74542894 194638446    28%    /home
/dev/md0           1012974      284    931654     0%    /tmp
bash-2.05b# swapinfo
Device          1K-blocks     Used    Avail Capacity
/dev/mirror/gm1   4194300      148  4194152     0%

I set the system up with a Makefile and some initial disk labels
(below my sig) and lots of trial and error, executing make zap, make
label_disks, etc.  No doubt less than perfect, but it worked for me.

At work, I've developed a makefile based version of RSE's "single disk
to two disk" approach.  If there is any interest, I'll ask my boss if I
can release it.

Is anybody doing anything about:
Making systinstall GEOM aware?
Putting RSE's excellent instructions in the handbook?

-- 
Adrian Wontroba

==> Makefile <==
clear:
	# stop and clear anything left over from last attempt
	@logger -p crit "clear"
	-gmirror stop -v gm3
	-gmirror clear -v stripe/gs0
	-gstripe stop -v gs0
	-gstripe clear -v ad8e ad10e
	-gmirror stop -v gm2
	-gmirror clear -v ad10d
	-gmirror stop -v gm1
	-gmirror clear -v ad8d
	-gmirror stop -v gm0 
	-gmirror clear -v ad8a ad10a

zap:
	# zap the front of the disks
	@logger -p crit "zap"
	dd if=/dev/zero of=/dev/ad8 count=100
	dd if=/dev/zero of=/dev/ad10 count=100

label_disks:
	# label disks
	@logger -p crit "label disks"
	fdisk -v -B -I /dev/ad8
	fdisk -v -B -I /dev/ad10
	bsdlabel -w -B ad8
	bsdlabel -w -B ad10

label_partitions:
	@logger -p crit "label slices"
	bsdlabel -R ad8 initial_ad8.label
	bsdlabel ad8 > ad8.label
	bsdlabel -R ad10 initial_ad10.label
	bsdlabel ad10 > ad10.label

mirrors:
	# bring on the mirrors
	@logger -p crit "gm0"
	gmirror label -v -h -n -b round-robin gm0 ad8a ad10a
	@logger -p crit "gm1"
	gmirror label -v -h -n -b round-robin gm1 ad8d
	@logger -p crit "gm2"
	gmirror label -v -h -n -b round-robin gm2 ad10d
	@logger -p crit "gs0"
	gstripe label -v -h gs0 ad8e ad10e
	@logger -p crit "gm3"
	gmirror label -v -h -n -b round-robin gm3 stripe/gs0

label_mirrors:
	# label the mirrors which contain more than one filesystem
	# gm2 contains d: var and e: usr
	bsdlabel -w mirror/gm2
	bsdlabel -R mirror/gm2 initial_gm2.label
	bsdlabel mirror/gm2 > gm2.label

newfs:
	# newfs the filesystems
	# gm1 is swap, doesn't need newfs
	newfs -U /dev/mirror/gm0 | tee gm0.newfs
	newfs -U /dev/mirror/gm2d | tee gm2d.newfs
	newfs -U /dev/mirror/gm2e | tee gm2e.newfs
	newfs -U /dev/mirror/gm3 | tee gm3.newfs

restore_root:
	# dump and restore root filesystem
	mount /dev/mirror/gm0 /mnt; dump -L -C 32 -0 -f- / | (cd /mnt; restore -r -f-)

restore_rest:
	# dump and restore other filesystem
	mount /dev/mirror/gm2d /mnt/var; dump -L -C 32 -0 -f- /var | (cd /mnt/var; restore -r -f-)
	mount /dev/mirror/gm2e /mnt/usr; dump -L -C 32 -0 -f- /usr | (cd /mnt/usr; restore -r -f-)
	mount /dev/mirror/gm3 /mnt/home; dump -L -C 32 -0 -f- /home | (cd /mnt/home; restore -r -f-)
	
zap1:
	# zap the front of the disks
	@logger -p crit "zap"
	dd if=/dev/zero of=/dev/ad4 count=100
	dd if=/dev/zero of=/dev/ad6 count=100

label_disks1:
	# label disks
	@logger -p crit "label disks"
	fdisk -v -B -I /dev/ad4
	fdisk -v -B -I /dev/ad6
	bsdlabel -w -B ad4
	bsdlabel -w -B ad6

label_partitions1:
	@logger -p crit "label slices"
	bsdlabel -R ad4 initial_ad4.label
	bsdlabel ad4 > ad4.label
	bsdlabel -R ad6 initial_ad6.label
	bsdlabel ad6 > ad6.label

mirrors1:
	# bring on the mirror plexes
	@logger -p crit "gm0"
	gmirror insert -v -h gm0 ad4a ad6a
	@logger -p crit "gm1"
	gmirror insert -v -h gm1 ad4d
	@logger -p crit "gm2"
	gmirror insert -v -h gm2 ad6d
	@logger -p crit "gs1"
	gstripe label -v -h gs1 ad4e ad6e
	@logger -p crit "gm3"
	gmirror insert -v -h gm3 stripe/gs1

auto1:
	# configure mirrors for automatic recovery
	gmirror configure -v -h -a gm0
	gmirror configure -v -h -a gm1
	gmirror configure -v -h -a gm2
	gmirror configure -v -h -a gm3

==> initial_ad10.label <==
# /dev/ad8:
8 partitions:
#        size   offset    fstype   [fsize bsize bps/cpg]
  c: 312581808        0    unused        0     0         # "raw" part, don't edit
  a:  1G        *    4.2BSD	        0     0       
  d:  4G * 4.2BSD 0     0       
  e:  * * 4.2BSD 0     0       

==> initial_ad10s1.label <==
# /dev/ad8s1:
8 partitions:
#        size   offset    fstype   [fsize bsize bps/cpg]
  a:  1G        *    4.2BSD	        0     0       
  c: 312581745        0    unused        0     0         # "raw" part, don't edit
  d:  4G * 4.2BSD 0     0       
  e:  * * 4.2BSD 0     0       

==> initial_ad4.label <==
# /dev/ad8:
8 partitions:
#        size   offset    fstype   [fsize bsize bps/cpg]
  c: 312581808        0    unused        0     0         # "raw" part, don't edit
  a:  1G        *    4.2BSD	        0     0       
  d:  4G * 4.2BSD 0     0       
  e:  * * 4.2BSD 0     0       

==> initial_ad6.label <==
# /dev/ad8:
8 partitions:
#        size   offset    fstype   [fsize bsize bps/cpg]
  c: 312581808        0    unused        0     0         # "raw" part, don't edit
  a:  1G        *    4.2BSD	        0     0       
  d:  4G * 4.2BSD 0     0       
  e:  * * 4.2BSD 0     0       

==> initial_ad8.label <==
# /dev/ad8:
8 partitions:
#        size   offset    fstype   [fsize bsize bps/cpg]
  c: 312581808        0    unused        0     0         # "raw" part, don't edit
  a:  1G        *    4.2BSD	        0     0       
  d:  4G * 4.2BSD 0     0       
  e:  * * 4.2BSD 0     0       

==> initial_ad8s1.label <==
# /dev/ad8s1:
8 partitions:
#        size   offset    fstype   [fsize bsize bps/cpg]
  a:  1G        *    4.2BSD	        0     0       
  c: 312581745        0    unused        0     0         # "raw" part, don't edit
  d:  4G * 4.2BSD 0     0       
  e:  * * 4.2BSD 0     0       

==> initial_gm1.label~ <==
# /dev/mirror/gm1:
8 partitions:
#        size   offset    fstype   [fsize bsize bps/cpg]
  a:  8388591       16    unused        0     0       
  c:  8388607        0    unused        0     0         # "raw" part, don't edit

==> initial_gm2.label <==
# /dev/mirror/gm1:
8 partitions:
#        size   offset    fstype   [fsize bsize bps/cpg]
  c:  8388607        0    unused        0     0         # "raw" part, don't edit
  d:  1G       *    4.2BSD	        0     0       
  e:  *       *    4.2BSD	        0     0       


More information about the freebsd-geom mailing list