GPT boot with ZFS RAIDZ "ZFS: i/o error - all block copies unavailable"

[Merijn Verstraaten]

On Tue, 27 Oct 2009 15:49:36 +0100, Robert Noland <rnoland at freebsd.org>  
wrote:
>> I tried again yesterday evening by recompiling RELENG_8 and -CURRENT. I
>> somehow managed to boot into RELENG_8 the first time, but after that  
>> this
>> error comes up:
>>
>>      ZFS: i/o error - all block copies unavailable
>>      ZFS: can't read object set for dataset ldd
>>      Can't find root filesystem - giving up
>>      ZFS: unexpected object set type ldd
>>      ZFS: unexpected object set type ldd
>>
>>      FreeBSD/i386 boot
>>      Default:/ tank:/boot/kernel/kernel
>>      boot:
>>      ZFS: unexpected object set type ldd
>>
>>      FreeBSD/i386 boot
>>      Default:/ tank:/boot/kernel/kernel
>>      boot: status
>>
>>       pool: tank
>>       config:
>>           NAME    STATE
>>           tank   ONLINE
>>         raidz1   ONLINE
>>          ad4p3   ONLINE
>>          ad6p3   ONLINE
>>          ad8p3   ONLINE
>>         ad10p3   ONLINE
>>
>> After recompiling world/kernel for -CURRENT I get roughly the same  
>> error:
>>
>>      ZFS: i/o error - all block copies unavailable
>>      ZFS: can't read object set for dataset 30
>>      Can't find root filesystem - giving up
>>      ZFS: unexpected object set type 0
>>      ZFS: unexpected object set type 0
>>
>>      FreeBSD/i386 boot
>>      Default:/ tank:/boot/kernel/kernel
>                ^^
> This looks strange... Do you have bootfs set to /, or something in
> loader.conf?  Does it work if you just type "tank:/boot/kernel/kernel"
> at this point?
>
> robert.

I think this might be user error. I just checked and the leading / is  
absent on my screen:

   ZFS: i/o error - all block copies unavailable
   ZFS: can't read object set for dataset 30
   Can't find root filesystem - giving up
   ZFS: unexpected object set type 0
   ZFS: unexpected object set type 0

   FreeBSD/i386 boot
   Default: tank:/boot/kernel/kernel
   boot:

I probably just typo'ed it this morning. As clarification I have  
vfs.root.mountfrom="zfs:tank/root" in loader.conf with my root filesystem  
installed on tank/root and then tank/usr, tank/var etc mounted on  
tank/root. If you need more detail my current setup procedure is:

"gpart create -s GPT <disk>"
"gpart add -b 34 -s 128 -t freebsd-boot <disk>"
"gpart add -b 162 -s 1G -t freebsd-swap <disk>"
"gpart add -t freebsd-zfs <disk>"

"zpool create tank raidz <disk1> <disk2> <diskN>"

"zfs set checksum=fletcher4 tank"

"zfs create -o reserv=512m tank/root"

"zfs create -o mountpoint=/tank/root/usr tank/usr"
"zfs create -o mountpoint=/tank/root/tmp tank/tmp"
"zfs create -o mountpoint=/tank/root/var tank/var"
"zfs create -o mountpoint=/tank/root/home tank/home"
"zfs create tank/usr/obj"
"zfs create tank/usr/src"
"zfs create tank/usr/ports"

export DESTDIR=/tank/root
Run the ./install.sh scripts in the various directories of the dist


"mkdir /boot/zfs"
"zpool export tank && zpool import tank"
"cp /boot/zfs/zpool.cache /tank/root/boot/zfs/"

Set 'LOADER_ZFS_SUPPORT=yes' in /tank/root/etc/src.conf

"chroot /tank/root"
"mount -t devfs devfs /dev"
"unset DESTDIR"
"cd /usr/src/sys/boot/"
"make obj"
"make depend"
"make"
"cd i386/loader"
"make install"
"umount /dev"
"exit"
"export LD_LIBRARY_PATH=/dist/lib"

"gpart bootcode -b /tank/root/boot/pmbr -p /tank/root/boot/gptzfsboot -i 1  
<disk>"

/boot/loader.conf:
     zfs_load="YES"
     vfs.root.mountfrom="zfs:tank/root"

/etc/rc.conf:
     zfs_enable="YES"

"zfs umount -a"
"zfs set mountpoint=legacy tank"
"zfs set mountpoint=/tmp tank/tmp"
"zfs set mountpoint=/var tank/var"
"zfs set mountpoint=/usr tank/usr"
"zfs set mountpoint=/home tank/home"
"zpool set bootfs=tank/root tank"

Then reboot, csup sources to whatever version to test and build  
world/kernel, install and watch things breaks.

Kind regards,
Merijn