Upgrade of RELENG_8 ZFS boot pool leads to unbootable system

[Paul Mather]

Yesterday, I updated my RELENG_8 ZFS-only system that has worked like a champ for ages.  After a successful install{kernel,world} and reboot, I noticed the 20121130 entry in /usr/src/UPDATING and upgraded my ZFS pool via "zfs upgrade -a".  I also upgraded my boot blocks as requested, and as per the "ZFS notes" section of /usr/src/UPDATING.

Unfortunately rebooting with the upgraded pool failed.  The "windmill" boot spinner spins for a tiny amount of time and then stops dead. :-(  I don't get to the boot loader menu at all.

I downloaded a very recent RELENG_8 snapshot (FreeBSD-8.3-RELENG_8-r244923-JPSNAP-amd64-amd64-memstick.img) from pub.allbsd.org and was able to boot successfully from USB using that.  I entered Fixit Mode and tried to write the boot blocks on the memstick image onto my hard drives but the resultant system still wouldn't boot.  The commands I used (from Fixit Mode) are these:

	gpart bootcode -b /dist/boot/pmbr -p /dist/boot/gptzfsboot -i 1 ad4
	gpart bootcode -b /dist/boot/pmbr -p /dist/boot/gptzfsboot -i 1 ad6

(ad4 and ad6 are my two hard drives.)

If I "load zfs" before booting the USB memstick then I can see my old pool listed when I do "zfs import".  I haven't tried importing the pool because I'm not sure if that would make the problem worse.

Does anyone have any advice in restoring this system to bootability?  I followed the standard "root on ZFS" recipe using a two drive mirror when installing the system initially.  Each drive uses GPT with three partitions: freebsd-boot, freebsd-swap, and freebsd-zfs in that order.  Like I said at the start, all this worked for a long time until just now when I upgraded the pool to enable "feature flags" support. :-(

Any help is appreciated.

Cheers,

Paul.