unable to boot a healthy zfs pool: all block copies unavailable

[Alan Somers]

On Mon, Nov 9, 2015 at 12:32 AM, Eugene M. Zheganin <emz at norma.perm.ru> wrote:
> Hi.
>
> On 06.11.2015 21:00, Alan Somers wrote:
>> I notice that my 10.2-RELEASE VM prints the same message about "all
>> block copies unavailable" and then continues to boot just fine.  So I
>> wonder if that part is just red herring.  There is another possibility
>> here: I have seen a bug where ZFS attempts to open the root pool's
>> vdevs by path (eg ada0p3) but can't find them because disks have been
>> replaced and no longer have their old devnames.  So vdev_geom searches
>> through the list of geom providers looking for any provider with the
>> correct ZFS GUID.  Normally it would find the right devname (eg
>> ada1p3).  But sometimes, because the disks are partitioned, it will
>> find the wrong partition first (eg ada1).  Since ZFS has labels at
>> both the beginning and the end of each vdev, vdev_geom will see the
>> label at the end of ada1 (really, it's the label at the end of ada1p3,
>> but it shares the same LBA that a label at the end of ada1 would) and
>> think that it opened ada1 successfully. vdev_geom_open will then
>> return, and at some later date another part of ZFS will fail to read
>> the MOS, and your boot will fail.
>>
> You are talking here about gptzfsboot being not smart enough, right ?
> Since kernel itself is able to find that pool after being booted up from
> alternative source. So it's a gptzfsboot issue, right ?
>
> Eugene.

Actually, no.  gptzfsboot successfully loaded the kernel, but the
kernel couldn't load the root zpool.  However, if the system had been
fully booted, then libzfs + kernel would've been able to load the
zpool.  The normal zpool import process consists of libzfs trying to
read all the vdev labels, then it tells the kernel which ones worked.
But when the kernel loads the root zpool it doesn't have any help from
libzfs.  That's why this bug only affects booting.

-Alan