Booting from ZFS raidz

[Stefan Bethke]

Am 17.12.2008 um 19:25 schrieb Doug Rabson:

> I've been working on adding raidz and raidz2 support to the boot  
> code and I have a patch which could use some testing if anyone here  
> is interested. This http://people.freebsd.org/~dfr/raidzboot-17122008.diff 
>  adds support for raidz and raidz2. The easiest way to prepare a  
> bootable pool is to put a GPT boot partition on each disk that will  
> make up the raidz pool and install gptzfsboot on the boot partition  
> of every drive.

Not sure I did things the right way, and it doesn't appear to be  
working correctly. I'm trying this in VMware Fusion, with three SCSI  
disks, which I configured like this:

Updated sources yesterday, then applied the patch and added  
LOADER_ZFS_SUPPORT?=YES to make.conf, then make buildworld buildkernel.

Created a GPT label and one partition on each of the three drives:

	gpart create -s gpt $1
	gpart add -b 34 -s 128 -t freebsd-boot $1
	gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 $1
	gpart add -b 512 -s 41900000 -t freebsd-zfs $1
	gpart list $1

(The disks are 20GB each)

root at freebsd-current:~# gpart list da3
Geom name: da3
fwheads: 255
fwsectors: 63
last: 41943006
first: 34
entries: 128
scheme: GPT
Providers:
1. Name: da3p1
    Mediasize: 65536 (64K)
    Sectorsize: 512
    Mode: r0w0e0
    rawtype: 83bd6b9d-7f41-11dc-be0b-001560b84f0f
    label: (null)
    length: 65536
    offset: 17408
    type: freebsd-boot
    index: 1
2. Name: da3p2
    Mediasize: 21452800000 (20G)
    Sectorsize: 512
    Mode: r1w1e1
    rawtype: 516e7cba-6ecf-11d6-8ff8-00022d09712b
    label: (null)
    length: 21452800000
    offset: 262144
    type: freebsd-zfs
    index: 2
Consumers:
1. Name: da3
    Mediasize: 21474836480 (20G)
    Sectorsize: 512
    Mode: r1w1e2


Created a raidz pool:
# zpool create tank raidz da1p2 da2p2 da3p2

Populated the filesystem with
# cd /usr/src && make installworld installkernel distribution DESTDIR=/ 
tank

Added zfs_load="YES" and vfs.root.mountfrom="zfs:tank" to loader.conf


When trying to boot, I get a number of "error 4 lba xxx", then "ZFS: i/ 
o error - all block copies are unavailable". The loader starts up, but  
cannot load /boot/loader.conf or /boot/device.hints. The LBA blocks  
are all towards the end of the disks, in the 4294626000 and up range.

Booted again from a different disk and ran zpool scrub; waited for  
that to complete without errors.

Next boot try now gives me (transcribed by hand):

ZFS: i/o error - all block copies unavailable
ZFS: can't read MOS
ZFS: unexpected object set type lld
ZFS: unexpected object set type lld

FreeBSD/i386 boot
Default: tank:/boot/kernel/kernel
boot:
ZFS: unexpected object set type lld

FreeBSD/i386 boot
Default: tank:/boot/kernel/kernel
boot:

Booting again from a different disk, running zpool status reveals no  
errors. Running scrub again, then next boot try.

root at freebsd-current:~# zpool scrub tank
root at freebsd-current:~# zpool status
   pool: tank
  state: ONLINE
  scrub: scrub in progress for 0h0m, 11.18% done, 0h0m to go
config:

	NAME        STATE     READ WRITE CKSUM
	tank        ONLINE       0     0     0
	  raidz1    ONLINE       0     0     0
	    da1p2   ONLINE       0     0     0
	    da2p2   ONLINE       0     0     0
	    da3p2   ONLINE       0     0     0

errors: No known data errors
root at freebsd-current:~# zpool status
   pool: tank
  state: ONLINE
  scrub: scrub completed after 0h0m with 0 errors on Fri Dec 19  
22:40:18 2008
config:

	NAME        STATE     READ WRITE CKSUM
	tank        ONLINE       0     0     0
	  raidz1    ONLINE       0     0     0
	    da1p2   ONLINE       0     0     0
	    da2p2   ONLINE       0     0     0
	    da3p2   ONLINE       0     0     0

errors: No known data errors

On the third boot try, same errors as on the second one.


Stefan

-- 
Stefan Bethke <stb at lassitu.de>   Fon +49 170 346 0140