gptzfsboot targeting wrong vdev

[Christian Kratzer]

Hi,

On Tue, 7 Jul 2020, Allan Jude wrote:

> On 2020-07-07 02:21, Andriy Gapon wrote:
>> On 06/07/2020 16:10, Christian Kratzer wrote:
<snipp/>
>>> When booting from ada0 I get following:
>>>
>>>     ZFS: i/o error - all block copies unavailable
>>>     ZFS: can't read MOS of pool zp1
>>>     gptzfsboot: failed to mount default pool zp1
>>>
>>>     FreeBSD/x86 boot
>>>
>
> So, just to be clear, at this point you have not loaded the boot loader
> yet. You are in the bootstrap (gptzfsboot), and it is unable to load the
> loader.


Sorry for the delay in answering.  I got around to some testing again.

Thanks for clarifying this. It does make a lot more sense now.


> I think it just looks at the first 'freebsd-zfs' type'd partition.
> However, if zp1 is GELI encrypted, it shouldn't be able to even tell the
> name of the pool.

That is what I have also been confused about.  It should not have access
to the zpool.cache and and should have no way of seeing zp1 which is
in geli encrypted da0p1 and da1p2.

The loader.conf contains the keys to those which come visbile when the
kernel starts.

So I was pretty confused on where it was getting the name zp1 from which
to my undestanding should not be visible anywhere.

> You might try changing the partition type of the boots you are not
> booting from, to 'freebsd-vinum' or something other than 'freebsd-zfs'
> so that gptzfsboot only sees 1 'freebsd-zfs' to boot from.


Before trying that I tried setting the bootme property on the root partitions
on ada0 and ada1.

That did not help either so I proceeded to set all but the zroot partition type to freebsd-vinum

 	root at zfs1:/home/ck # gpart show
 	=>        40  7814037088  da0  GPT  (3.6T)
 		  40        4056       - free -  (2.0M)
 		4096  7814029312    1  freebsd-vinum  (3.6T)
 	  7814033408        3720       - free -  (1.8M)

 	=>        40  7814037088  da1  GPT  (3.6T)
 		  40        4056       - free -  (2.0M)
 		4096  7814029312    1  freebsd-vinum  (3.6T)
 	  7814033408        3720       - free -  (1.8M)

 	=>        40  7814037088  da2  GPT  (3.6T)
 		  40        4056       - free -  (2.0M)
 		4096  7814029312    1  freebsd-vinum  (3.6T)
 	  7814033408        3720       - free -  (1.8M)

 	=>        40  7814037088  da3  GPT  (3.6T)
 		  40        4056       - free -  (2.0M)
 		4096  7814029312    1  freebsd-vinum  (3.6T)
 	  7814033408        3720       - free -  (1.8M)

 	=>        40  7814037088  da4  GPT  (3.6T)
 		  40        4056       - free -  (2.0M)
 		4096  7814029312    1  freebsd-vinum  (3.6T)
 	  7814033408        3720       - free -  (1.8M)

 	=>        40  7814037088  da5  GPT  (3.6T)
 		  40        4056       - free -  (2.0M)
 		4096  7814029312    1  freebsd-vinum  (3.6T)
 	  7814033408        3720       - free -  (1.8M)

 	=>        40  7814037088  da6  GPT  (3.6T)
 		  40        4056       - free -  (2.0M)
 		4096  7814029312    1  freebsd-vinum  (3.6T)
 	  7814033408        3720       - free -  (1.8M)

 	=>        40  7814037088  da7  GPT  (3.6T)
 		  40        4056       - free -  (2.0M)
 		4096  7814029312    1  freebsd-vinum  (3.6T)
 	  7814033408        3720       - free -  (1.8M)

 	=>       40  468862048  ada0  GPT  (224G)
 		 40       1024     1  freebsd-boot  (512K)
 	       1064  134217728     2  freebsd-swap  (64G)
 	  134218792   33554432     3  freebsd-vinum  (16G)
 	  167773224   33554432     4  freebsd-vinum  (16G)
 	  201327656  267534424     5  freebsd-zfs  [bootme]  (128G)
 	  468862080          8        - free -  (4.0K)

 	=>       40  468862048  ada1  GPT  (224G)
 		 40       1024     1  freebsd-boot  (512K)
 	       1064  134217728     2  freebsd-swap  (64G)
 	  134218792   33554432     3  freebsd-vinum  (16G)
 	  167773224   33554432     4  freebsd-vinum  (16G)
 	  201327656  267534424     5  freebsd-zfs  [bootme]  (128G)
 	  468862080          8        - free -  (4.0K)

That helped.  The system now finally boots from the mirror on ada0p5 and ada1p5.


Unclear why setting the bootme property did not help on its own.


Thanks 
Christian

-- 
Christian Kratzer                   CK Software GmbH
Email:   ck at cksoft.de               Wildberger Weg 24/2
Phone:   +49 7032 893 997 - 0       D-71126 Gaeufelden
Fax:     +49 7032 893 997 - 9       HRB 245288, Amtsgericht Stuttgart
Mobile:  +49 171 1947 843           Geschaeftsfuehrer: Christian Kratzer
Web:     http://www.cksoft.de/