12.1p7 no longer boots after doing zpool upgrade -a
John Kennedy
warlock at phouka.net
Thu Jul 9 21:20:17 UTC 2020
On Thu, Jul 09, 2020 at 08:24:54AM -0500, Kyle Evans wrote:
> On Thu, Jul 9, 2020 at 8:12 AM Guido van Rooij <guido at gvr.org> wrote:
> >
> > I did a zpool upgrade -a to enable large_dnode and spacemap_v2.
> > After that, I did:
> > gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 2 ada0
> > gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 2 ada1
> > and:
> > gpart bootcode -p /boot/boot1.efifat -i 1 ada0
> > gpart bootcode -p /boot/boot1.efifat -i 1 ada1
> >
> > Now the system no longer boots from either disk and drops to the efi shell.
>
> This method of updating the ESP is no longer recommended for new 12.x
> installations -- we now more carefully construct the ESP with an
> /EFI/FreeBSD/loader.efi where loader.efi is /boot/loader.efi. You will
> want to rebuild this as such, and that may fix part of your problem.
So, I've got a stable box:
FreeBSD 12.1-STABLE #156 r362983+cd7a9efd329c(stable/12): Tue Jul 7 07:42:17 PDT 2020
I've got almost the same situation, except I haven't shot myself in the foot yet.
My partitions:
# gpart show nvd0 nvd1
=> 40 976773088 nvd0 GPT (466G)
40 409600 1 efi (200M)
409640 1024 2 freebsd-boot (512K)
410664 984 - free - (492K)
411648 16777216 3 freebsd-swap (8.0G)
17188864 959584256 4 freebsd-zfs (458G)
976773120 8 - free - (4.0K)
=> 40 976773088 nvd1 GPT (466G)
40 409600 1 efi (200M)
409640 1024 2 freebsd-boot (512K)
410664 984 - free - (492K)
411648 16777216 3 freebsd-swap (8.0G)
17188864 959584256 4 freebsd-zfs (458G)
976773120 8 - free - (4.0K)
So this is my first UEFI system that's hit an upgrade point:
# zpool status
pool: zroot
state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
the pool may no longer be accessible by software that does not support
the features. See zpool-features(7) for details.
scan: scrub repaired 0 in 0 days 00:02:04 with 0 errors on Wed Jul 1 13:21:57 2020
config:
NAME STATE READ WRITE CKSUM
zroot ONLINE 0 0 0
nvd1p4.eli ONLINE 0 0 0
nvd0p4.eli ONLINE 0 0 0
I usually seed these things off of stick and they evolve from there, but this
one is slightly interesting because the motherboard died repeatedly and I'm
guessing that the M.2 drives got reinstalled in the "wrong" order so now if I
don't manually select one of the M.2 drives, it won't reboot (acts like the
encryption password is bad, fails out into a shell where I reboot).
In any case, I know if I upgrade, it is going to tell me to upgrade the boot
code and I'm wondering what that is these days.
If this was a Pre-UEFI, I would be expecting to do (only) something like:
gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 2 nvd0
gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 2 nvd1
IE, might hit either drive, update the freebsd-boot partition by # on both,
and special note that the # is now one higher because of the EFI partition.
The FreeBSD wiki still has the "old" way of doing it here...
https://wiki.freebsd.org/RootOnZFS/GPTZFSBoot
... but your newer way of doing it here:
https://wiki.freebsd.org/UEFI
So the newer way of doing it would be something like this, except I'd by updating
the file rather than looking at it's checksum/version?
strings -a < /boot/loader.efi | grep -A1 'EFI loader'
FreeBSD/amd64 EFI loader, Revision 1.1
(Tue Jul 7 07:33:24 PDT 2020 warlock at ouroboros.phouka.net)
mount -t msdosfs /dev/nvd0p1 /mnt
md5 -r /boot/loader.efi /mnt/efi/boot/BOOTx64.efi
9f64a96f3170e3327ca7dad1d574d852 /boot/loader.efi
cbff58cc8ee79ce7342f5be23ad14ba3 /mnt/efi/boot/BOOTx64.efi
strings -a < /mnt/efi/boot/BOOTx64.efi | grep -A1 'EFI loader'
FreeBSD/amd64 EFI loader, Revision 1.1
|/-\
umount -v /mnt
mount -t msdosfs /dev/nvd1p1 /mnt
md5 -r /mnt/efi/boot/BOOTx64.efi
cbff58cc8ee79ce7342f5be23ad14ba3 /mnt/efi/boot/BOOTx64.efi
umount -v /mnt
At least some of the checksum difference is because WITHOUT_REPRODUCIBLE_BUILD=YES.
(Sorry for extra details, but I know when I was researching this, I ran
across a *lot* of examples where we didn't have a GPT dump and people
were just using partition #s and if you stomp on the wrong partition
you're going to have a bad day, so leaving some specific examples here.)
So one recipe doesn't even seem to make the freebsd-boot partition, so is it
optional for a pure UEFI boot? Should we always gpart-bootcode it if it exists
and upgrade BOOTx64.efi when the EFI partition exists, or is there a few more
wrinkles we need to worry about?
In any case, is it a logical theory that my possible boot-order problem
is because drive order got swapped and maybe one wasn't properly updated?
They seem to be the same:
# md5 /dev/nvd[01]p2
MD5 (/dev/nvd0p2) = 2ded438a2c97ea551446cc2d1d3b498e
MD5 (/dev/nvd1p2) = 2ded438a2c97ea551446cc2d1d3b498e
Ideally I'd like to have not boot through the UEFI boot menu every time.
I'm not sure why the drive order seems to matter right now.
More information about the freebsd-stable
mailing list