handbook incorrect in 19.4 RAID1 for ia64

[Anton Shterenlikht]

After careful reading section 19.4 RAID1 of the handbook,
and after some discussion on ia64 list, I believe this
section is incorrect when it comes to ia64 platform.

I'd like to write a correction, but not sure if I should
aim for a subsection, e.g  "19.4.x RAID1 on ia64", or
just some notes and examples in the main 19.4 article.

The major problems are:

(a) ia64 uses GPT instead of MBR, and secondary GPT
is stored in the last 34 sectors of the disk, so
that it is damaged by gmirror;

(b) because of (a) gmirror per partition is the only
possible way forward.

(c) gmirror per partition ivolves serveral extra steps
because only unmounted partition can be mirrored.


My experience is briefly detailed below, and I was
thinking of submitting something along these lines.

Please advise 
anton

*******************

Some steps are related to install on my particular box,
i.e. not generic ia64 requirements.

It seems quite complicated, requiring two reboots.
Please see if you can shorten or simplify the procedure.

I'd like to write a section on RAID-1 on ia64 for the
handbook based on this, because existing advice is
incorrect when it comes to ia64. So, I welcome any
suggestions on these quidelines.

This example is based on setting up gmirror on
rx2600 with two directly attached scsi disks, da0 and da1.

1. preliminary: read man pages for
	gpart(8), gmirror(8), geom(4), geom(8), dump(8), restore(8).

	use

	# gpart show
	# gmirror status
	# gmirror list

	between the steps to see partitions and mirror state.

2. fresh FBSD install on da0, as default, but:
	a) /efi is 400MB
	b) set up at least one network interface
	c) create at least one user and add him to group wheel

	(b) and (c) are necessary because /etc/ttys by default
	does not open getty on console via MP, so ssh connection
	is required.

3. (optional, possibly for my box only):
	 add hw.ata.atapi_dma=0 to /boot/device.hints

4. (optional) enable extra debug messages:
	# sysctl kern.geom.debugflags=17

5. partition a spare disk, da1, exactly as the boot disk:
	# gpart create -s gpt da1
	# gpart add ...

	so that the results look like:

	# gpart show
	=>       34  143374671  da0  GPT  (68G)
	         34     819200    1  efi  (400M)
	     819234    1048576    2  freebsd-ufs  (512M)
	    1867810    4194304    3  freebsd-swap  (2.0G)
	    6062114    2097152    4  freebsd-ufs  (1.0G)
	    8159266    2097152    5  freebsd-ufs  (1.0G)
	   10256418  133118287    6  freebsd-ufs  (63G)

	=>       34  143374671  da1  GPT  (68G)
	         34     819200    1  efi  (400M)
	     819234    1048576    2  freebsd-ufs  (512M)
	    1867810    4194304    3  freebsd-swap  (2.0G)
	    6062114    2097152    4  freebsd-ufs  (1.0G)
	    8159266    2097152    5  freebsd-ufs  (1.0G)
	   10256418  133118287    6  freebsd-ufs  (63G)

	#

6. load gmirror kernel module
	# gmirror load

7. create mirror for EFI partition:
	a) unmount /efi because GEOM manipulations can be
	performed only on unmounted, not in use, partition:
	# umount /efi

	b) create efi mirror on the boot(!) disk:
	# gmirror label -vb round-robin efi da0p1

	c) add spare disk's efi partition to the mirror:
	# gmirror insert efi da1p1

	rebuilding should start, check progress with
	# gmirror status

	after rebuilding is complete you should see: 

	# gmirror status efi
	      Name    Status  Components
	mirror/efi  COMPLETE  da0p1
	                      da1p1
	#

	d) mount mirror efi:
	# mount -t msdosfs /dev/mirror/efi /efi

	# df
	Filesystem       1K-blocks    Used    Avail Capacity  Mounted on
	...
	/dev/mirror/efi     409504  163264   246240    40%    /efi
	#

8. create mirror for root (/) partition. This involves
extra steps since / cannot be unmounted.

	a) create mirror on the spare disk:
	# gmirror label -vb round-robin root da1p2

	b) create ufs filesystem on the mirror:
	# newfs /dev/mirror/root

	c) mount root mirror temporarily, say under /mnt:
	# mount /dev/mirror/root /mnt

	d) now copy / onto /mnt (actually onto the root mirror,
	/dev/mirror/root). Use a combination of dump(8) and
	restore(8). No other copying tool will do it right:
	# cd /mnt
	# dump 0aLf - / | restore rf -

	(check the man pages for more details on the options)

9. update fstab on the mirror(!) Edit /mnt/etc/fstab and change
	da0p1 into mirror/efi
	 and
	da0p2 into mirror/root.

10. update /boot/loader.conf

	a) put this line at the very beginning:
	geom_mirror_load="YES"

	f) In this line:
	vfs.root.mountfrom="ufs:/dev/da0p2"

	replace da0p2 with mirror/root, so that it is:
  
	vfs.root.mountfrom="ufs:/dev/mirror/root"

11. reboot into single user(!) mode
	on shutdown you should see root and efi mirrors
	destroyed.

12. on boot you should see gmirror loaded, /dev/mirror/root
and /dev/mirror/efi started and /dev/mirror/root is used
as a boot device,  

13. now that you have booted from /dev/da1p2, da0p2 is not
mounted, so it can be inserted into root mirror:

	# gmirror insert root da0p2

14. create mirrors for all other partitions of da0, which are
now not mounted:

	# gmirror label -vb round-robin swap da0p3
	# gmirror label -vb round-robin var da0p4
	# gmirror label -vb round-robin tmp da0p5
	# gmirror label -vb round-robin usr da0p6

15. edit /etc/fstab and change
	da0p3 into mirrior/swap	
	da0p4 into mirrior/var
	da0p5 into mirrior/tmp
	da0p6 into mirrior/usr

16. reboot

17. on startup add spare disks to mirrors:

	# gmirror insert swap da1p3
	# gmirror insert var da1p4
	# gmirror insert tmp da1p5
	# gmirror insert usr da1p6

18. done!

-- 
Anton Shterenlikht
Room 2.6, Queen's Building
Mech Eng Dept
Bristol University
University Walk, Bristol BS8 1TR, UK
Tel: +44 (0)117 928 8233 
Fax: +44 (0)117 929 4423