Re: zpool upgrade "don't forget to update boot code"

On 7/31/22 02:23, David Christensen wrote:

 > FreeBSD-12.1-RELEASE-amd64-memstick.img

 > Partitioning            Auto (ZFS)
 > Pool Type/Disks         stripe: 1 disk
 >                          ada0 <redacted>
 > Pool Name               soho2_zroot
 > Force 4K Sectors        YES
 > Encrypt Disks           YES
 > Partition Scheme        MBR (BIOS)
 > Swap Size               2g
 > Mirror Swap             YES
 > Encrypt Swap            YES
 >  >>> Install             Proceed with installation
 > Encryption passphrase   *******

 > # gpart show -p ada0
 > =>       63  117231345    ada0  MBR  (56G)
 >           63       1985          - free -  (993K)
 >         2048   29360128  ada0s1  freebsd  [active]  (14G)
 >
 > # gpart show -p ada0s1
 > =>       0  29360128   ada0s1  BSD  (14G)
 >           0   4194304  ada0s1a  freebsd-zfs  (2.0G)
 >     4194304   4194304  ada0s1b  freebsd-swap  (2.0G)
 >     8388608  20971520  ada0s1d  freebsd-zfs  (10G)

 > # zpool upgrade soho2_zroot
 > This system supports ZFS pool feature flags.
 >
 > Enabled the following features on 'soho2_zroot':
 >    allocation_classes
 >
 > If you boot from pool 'soho2_zroot', don't forget to update boot code.
 > Assuming you use GPT partitioning and da0 is your boot disk
 > the following command will do it:
 >
 >      gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da0


STFW here is information about MBR:

https://en.wikipedia.org/wiki/Master_boot_record


Crawling a recent FreeBSD installer (FreeBSD12.3-RELEASE-amd64-memstick) 
/usr/libexec/bsdinstall/zfsboot to see how it sets up boot code for MBR 
partitioning, ZFS boot, and encrypted ZFS root:

DD_WITH_OPTIONS='dd if="%s" of="%s" %s'

GPART_BOOTCODE='gpart bootcode -b "%s" "%s"'

zfs_create_diskpart()
{

	case "$ZFSBOOT_PARTITION_SCHEME" in

	MBR) f_dprintf "$funcname: Creating MBR layout..."

		#
		# 1. Create MBR layout (no labels)
		#

		f_eval_catch $funcname gpart "$GPART_BOOTCODE" /boot/mbr \
		             $disk || return $FAILURE


AIUI the above runs the following command:

# gpart bootcode -b /boot/mbr ada0


So, gpart(8) installed the FreeBSD MBR partitioning scheme bootloader 
into the OS drive master boot record bootstrap code area.


AIUI this fulfills FreeBSD boot process stage 0.


Looking at /boot/mbr, this file is before my installation date.  So, I 
do not need to update:

# ll /boot/mbr
-r--r--r--  1 root  wheel  512 2019/10/31 21:29:42 /boot/mbr


Continuing the crawl:

		#
		# 5. Add freebsd-zfs partition for zroot
		#

		f_eval_catch $funcname dd "$DD_WITH_OPTIONS" \
		             /boot/zfsboot /dev/${disk}s1 count=1 ||
		             return $FAILURE
		;;

	esac # $ZFSBOOT_PARTITION_SCHEME


AIUI the above runs the following command:

# dd if=/boot/zfsboot of=/dev/ada0s1 count=1


So, dd(1) installed the first block of /boot/zfsboot into the first 
block of ada0s1.


AIUI this fulfills FreeBSD boot process stage 1.


Looking at /boot/zfsboot, this file is after my installation date.

# ll /boot/zfsboot
-r--r--r--  1 root  wheel  262656 2022/01/30 20:51:01 /boot/zfsboot


Comparing, they are identical.  So, I do not need to update:


2022-07-31 23:45:51 toor@f3 ~
# cmp -n 512 /boot/zfsboot /dev/ada0s1

2022-07-31 23:46:48 toor@f3 ~
# echo $?
0


Continuing the crawl:

	case "$ZFSBOOT_PARTITION_SCHEME" in

	MBR) f_dprintf "$funcname: Creating MBR layout..."

		# NB: zpool will use s1a (no labels)
		bootpart=s1a swappart=s1b targetpart=s1d mbrindex=4

	esac # $ZFSBOOT_PARTITION_SCHEME


zfs_create_boot()
{

	# MBR boot loader touch-up
	if [ "$ZFSBOOT_PARTITION_SCHEME" = "MBR" ]; then

		f_dprintf "$funcname: Updating MBR boot loader on disks..."
		# Stick the ZFS boot loader in the "convenient hole" after
		# the ZFS internal metadata
		for disk in $disks; do
			f_eval_catch $funcname dd "$DD_WITH_OPTIONS" \
			             /boot/zfsboot /dev/$disk$bootpart \
			             "skip=1 seek=1024" || return $FAILURE
		done


AIUI the above runs the following command:

# dd if=/boot/zfsboot of=/dev/ada0s1a skip=1 seek=1024


So, dd(1) installed the second and remaining blocks of /boot/zfsboot 
into ada0s1a starting at byte offset 1024.


AIUI this fulfills FreeBSD boot process stage 2.


Comparing, they differ:

2022-07-31 23:59:07 toor@f3 ~
# cmp -i 512:1024 /boot/zfsboot /dev/ada0s1a
/boot/zfsboot /dev/ada0s1a differ: char 1, line 1

2022-08-01 00:04:22 toor@f3 ~
# hexdump -C -s 512 -n 16 /boot/zfsboot
00000200  eb 0e 42 54 58 01 02 80  f6 0f b0 06 00 20 00 00 
|..BTX........ ..|
00000200  eb 0e 42 54 58 01 02 80  f6 0f b0 06 00 20 00 00 
|..BTX........ ..|
00000210

2022-08-01 00:05:02 toor@f3 ~
# hexdump -C -s 1024 -n 16 /dev/ada0s1a
00000400  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00 
|................|
00000400  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00 
|................|
00000410


So, to "update boot code" after upgrading my ZFS boot and root pools, I 
need to repeat the stage 2 step performed by the FreeBSD installer for 
my configuration:

# dd if=/boot/zfsboot of=/dev/ada0s1a skip=1 seek=1024


Can anybody confirm the above analysis and conclusion before I try it?


David