gptzfsboot targeting wrong vdev

[Allan Jude]

On 2020-07-13 06:02, Christian Kratzer wrote:
> Hi,
> 
> On Tue, 7 Jul 2020, Allan Jude wrote:
> 
>> On 2020-07-07 02:21, Andriy Gapon wrote:
>>> On 06/07/2020 16:10, Christian Kratzer wrote:
> <snipp/>
>>>> When booting from ada0 I get following:
>>>>
>>>>     ZFS: i/o error - all block copies unavailable
>>>>     ZFS: can't read MOS of pool zp1
>>>>     gptzfsboot: failed to mount default pool zp1
>>>>
>>>>     FreeBSD/x86 boot
>>>>
>>
>> So, just to be clear, at this point you have not loaded the boot loader
>> yet. You are in the bootstrap (gptzfsboot), and it is unable to load the
>> loader.
> 
> 
> Sorry for the delay in answering.  I got around to some testing again.
> 
> Thanks for clarifying this. It does make a lot more sense now.
> 
> 
>> I think it just looks at the first 'freebsd-zfs' type'd partition.
>> However, if zp1 is GELI encrypted, it shouldn't be able to even tell the
>> name of the pool.
> 
> That is what I have also been confused about.  It should not have access
> to the zpool.cache and and should have no way of seeing zp1 which is
> in geli encrypted da0p1 and da1p2.
> 
> The loader.conf contains the keys to those which come visbile when the
> kernel starts.
> 
> So I was pretty confused on where it was getting the name zp1 from which
> to my undestanding should not be visible anywhere.
> 
>> You might try changing the partition type of the boots you are not
>> booting from, to 'freebsd-vinum' or something other than 'freebsd-zfs'
>> so that gptzfsboot only sees 1 'freebsd-zfs' to boot from.
> 
> 
> Before trying that I tried setting the bootme property on the root
> partitions
> on ada0 and ada1.
> 
> That did not help either so I proceeded to set all but the zroot
> partition type to freebsd-vinum
> 
>     root at zfs1:/home/ck # gpart show
>     =>        40  7814037088  da0  GPT  (3.6T)
>           40        4056       - free -  (2.0M)
>         4096  7814029312    1  freebsd-vinum  (3.6T)
>       7814033408        3720       - free -  (1.8M)
> 
>     =>        40  7814037088  da1  GPT  (3.6T)
>           40        4056       - free -  (2.0M)
>         4096  7814029312    1  freebsd-vinum  (3.6T)
>       7814033408        3720       - free -  (1.8M)
> 
>     =>        40  7814037088  da2  GPT  (3.6T)
>           40        4056       - free -  (2.0M)
>         4096  7814029312    1  freebsd-vinum  (3.6T)
>       7814033408        3720       - free -  (1.8M)
> 
>     =>        40  7814037088  da3  GPT  (3.6T)
>           40        4056       - free -  (2.0M)
>         4096  7814029312    1  freebsd-vinum  (3.6T)
>       7814033408        3720       - free -  (1.8M)
> 
>     =>        40  7814037088  da4  GPT  (3.6T)
>           40        4056       - free -  (2.0M)
>         4096  7814029312    1  freebsd-vinum  (3.6T)
>       7814033408        3720       - free -  (1.8M)
> 
>     =>        40  7814037088  da5  GPT  (3.6T)
>           40        4056       - free -  (2.0M)
>         4096  7814029312    1  freebsd-vinum  (3.6T)
>       7814033408        3720       - free -  (1.8M)
> 
>     =>        40  7814037088  da6  GPT  (3.6T)
>           40        4056       - free -  (2.0M)
>         4096  7814029312    1  freebsd-vinum  (3.6T)
>       7814033408        3720       - free -  (1.8M)
> 
>     =>        40  7814037088  da7  GPT  (3.6T)
>           40        4056       - free -  (2.0M)
>         4096  7814029312    1  freebsd-vinum  (3.6T)
>       7814033408        3720       - free -  (1.8M)
> 
>     =>       40  468862048  ada0  GPT  (224G)
>          40       1024     1  freebsd-boot  (512K)
>            1064  134217728     2  freebsd-swap  (64G)
>       134218792   33554432     3  freebsd-vinum  (16G)
>       167773224   33554432     4  freebsd-vinum  (16G)
>       201327656  267534424     5  freebsd-zfs  [bootme]  (128G)
>       468862080          8        - free -  (4.0K)
> 
>     =>       40  468862048  ada1  GPT  (224G)
>          40       1024     1  freebsd-boot  (512K)
>            1064  134217728     2  freebsd-swap  (64G)
>       134218792   33554432     3  freebsd-vinum  (16G)
>       167773224   33554432     4  freebsd-vinum  (16G)
>       201327656  267534424     5  freebsd-zfs  [bootme]  (128G)
>       468862080          8        - free -  (4.0K)
> 
> That helped.  The system now finally boots from the mirror on ada0p5 and
> ada1p5.
> 
> 
> Unclear why setting the bootme property did not help on its own.
> 
> 
> Thanks Christian
> 

bootme is only used by gptboot, for UFS.

You might want to example your disks with 'zdb -l /dev/ada0' (for each
disk, and each partition). It seems there is something not encrypted
somewhere. It may be an old label from a pool that has been destroyed,
and a new pool with the same name was later created or something.

-- 
Allan Jude

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 834 bytes
Desc: OpenPGP digital signature
URL: <http://lists.freebsd.org/pipermail/freebsd-fs/attachments/20200713/b222d720/attachment.sig>