misc/148655: [zfs] Booting from a degraded raidz no longer works in 8-STABLE [regression]

[Emil Smolenski]

>Number:         148655
>Category:       misc
>Synopsis:       [zfs] Booting from a degraded raidz no longer works in 8-STABLE [regression]
>Confidential:   no
>Severity:       serious
>Priority:       low
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Thu Jul 15 23:00:04 UTC 2010
>Closed-Date:
>Last-Modified:
>Originator:     Emil Smolenski
>Release:        FreeBSD 8-STABLE
>Organization:
>Environment:
FreeBSD 8.0-STABLE i386
>Description:
After upgrade 8.0-RELEASE system to 8-STABLE, the machine no longer boots from degraded raidz. Booting stops with a message:

error 1 lba 32
error 1 lba 1

Booting from a non-degraded raidz still works fine.

After encountering a problem I prepared an isolated (qemu) environment to test this issue.

In 8.0-RELEASE everything works fine. I can remove one disk and the system boots. My configuration:

- gpart config:
=>     34  6291389  ad0  GPT  (3.0G)
       34      128    1  freebsd-boot  (64K)
      162  6291261    2  freebsd-zfs  (3.0G)

=>     34  6291389  ad1  GPT  (3.0G)
       34      128    1  freebsd-boot  (64K)
      162  6291261    2  freebsd-zfs  (3.0G)

=>     34  6291389  ad3  GPT  (3.0G)
       34      128    1  freebsd-boot  (64K)
      162  6291261    2  freebsd-zfs  (3.0G)

(with GPT labeling)

- zpool (v13) config:
        NAME            STATE     READ WRITE CKSUM
        bijou           ONLINE       0     0     0
          raidz1        ONLINE       0     0     0
            gpt/bijou0  ONLINE       0     0     0
            gpt/bijou1  ONLINE       0     0     0
            gpt/bijou2  ONLINE       0     0     0

- loader.conf:
zfs_load="YES"
vfs.root.mountfrom="zfs:bijou"

- rc.conf:
zfs_enable="YES"

After upgrade to 8-STABLE (world and kernel) and BEFORE reinstalling bootcode booting from degraded raidz no longer works: http://img28.imageshack.us/img28/9118/raidz8stable.png . Note: the installworld upgrades /boot/loader.

After reinstalling bootcode:

gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ad0
gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ad1
gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ad3

booting stops with a message:

error 1 lba 32
error 1 lba 1

( http://img17.imageshack.us/img17/36/raidz8stablereinstalled.png )

Upgrading zpool to v14 doesn't help. I think this is serious regression -- disks often fail during reboot. The whole idea of raidz doesn't make any sense.
>How-To-Repeat:
1. Have FreeBSD 8.0-RELEASE "RootOnZFS" raidz1 installation (like the one described here: http://wiki.freebsd.org/RootOnZFS/GPTZFSBoot/RAIDZ1 )
2. Upgrade to 8-STABLE.
3. Physically remove one disk from raidz.
4. Boot the machine.
>Fix:


>Release-Note:
>Audit-Trail:
>Unformatted: