ZFSROOT UEFI boot

[Tomoaki AOKI]

Unfortunately, this (and its committed successor and original for UFS)
fails to boot in some situation, like below. OTOH, gptzfsboot (and
maybe gptboot for UFS, too) is OK.

When I select Disk1 from UEFI firmware, bootx64.efi in Disk1 EFI
partition is used and it searches /boot/loader.efi from Disk2 (in ZFS,
if none, in UFS) only.
And when I select Disk2, bootx64.efi in Disk2 EFI partition is used and
it searches /boot/loader.efi from Disk1 only.


In fact, this is a long-standing and living problem.
At past, USB memstick with head memstick.img (UEFI enabled, but
without root-on-ZFS support) booted fine, but after I added UFS2
partition in internal disk, the USB memstick didn't boot anymore.
It searches /boot/loader.efi from internal UFS and fails as it was
blank (only newfs'ed) at that time. Another USB memstick with stable/10
memstick.img is still fine, as it's still ancient BIOS based.

Possibly, it's not a fault of boot1.efi but caused by differense in
implementation of UEFI firmware. If that's it, different boot1.efi
would be needed for each implementation.

A bit more details of tests are as below. Not all combinations are
covered, but would be sufficient to determine above conclusion.


Common configurations for all tests:
  *Each disk has one EFI partition (p1), one freebsd-boot partition
   (p2), one swap partition (p3), one UFS partition (p4), and one
   ZFS pool (p5) with this order.

  *Each partition has different GEOM label.

  *In each disk, FreeBSD is installed as root on ZFS. No other OS.

  *stable/10 (r294614) is installed in Disk1.

  *head (r294567) is installed in Disk2.

  *ZFS-enabled boot1.efi (head r294567) is used as bootx64.efi.


Set 1: Boot from Disk1 (select it in UEFI firmware).
       In all tests, /boot/loader.efi in Disk1 (both UFS and ZFS)
       are NOT searched at all.

 1-1) Both UFS and ZFS has no /boot/loader.efi
        -> Fail to boot. Fall back to boot1 prompt.

 1-2) Disk2 UFS only has /boot/loader.efi, whole /boot of Disk2 ZFS
      is copied to UFS.
        -> head in Disk2 boots fine.

 1-3) Same as 1-2, except its /boot/loader.efi is overwritten by the
      one of stable/10.
        -> head in Disk2 boots fine, as loader.efi loads kernel from
           /boot/kernel/kernel in UFS and kernel with zfs.ko can mount
           root on ZFS specified by vfs.root.mountfrom.

 1-4) Disk2 UFS only has /boot/loader.efi, whole /boot of Disk1 ZFS
      is copied to UFS and its /boot/loader.efi is overwritten by
      the one of head.
        -> stable/10 in Disk1 ZFS boots fine.

 1-5) Disk2 ZFS only has /boot/loader.efi.
        -> head in Disk2 ZFS boots fine.

 1-6) Both UFS and ZFS in Disk2 has /boot/loader.efi.
      (Mix of 1-4 and 1-5)
        -> head in Disk2 ZFS boots fine.


Set 2: Boot from Disk2 (select it in UEFI firmware).
       In all tests, /boot/loader.efi in Disk2 (both UFS and ZFS)
       are NOT searched at all.

 2-1) Both UFS and ZFS has no /boot/loader.efi
        -> Fail to boot. Fall back to boot1 prompt.
           ZFS pool in Disk2 is shown before one in Disk1.

 2-2) Disk1 UFS only has /boot/loader.efi, whole /boot of Disk2 ZFS
      is copied to UFS.
        -> head in Disk2 ZFS boots fine.

 2-3) Disk1 UFS only has /boot/loader.efi, whole /boot of Disk1 ZFS
      is copied to UFS.
        -> stable/10 in Disk1 ZFS boots fine, as loader.efi loads
           kernel from /boot/kernel/kernel in UFS and kernel with zfs.ko
           can mount root on ZFS specified by vfs.root.mountfrom.

 2-4) Disk1 UFS only has /boot/loader.efi, whole /boot of Disk1 ZFS
      is copied to UFS and its /boot/loader.efi is overwritten by
      the one of head.
        -> stable/10 in Disk1 ZFS boots fine.

 2-5) Disk1 ZFS only has /boot/loader.efi of stable/10 itself.
        -> Fail to boot. Fall back to boot1 prompt.
           ZFS pool in Disk2 is shown before one in Disk1.

 2-6) Disk1 ZFS only has /boot/loader.efi of head.
        -> stable/10 in Disk1 ZFS boots fine.

 2-7) Both UFS and ZFS in Disk1 has /boot/loader.efi of head.
      (Mix of 2-2 and 2-6)
        -> stable/10 in Disk1 ZFS boots fine.

 2-8) UFS has /boot/loader.efi of head (head kernel copied), but ZFS
      has /boot/loader.efi of stable/10 itself. (Mix of 2-2 and 2-5)
        -> Same as 2-5. Fail to boot. Fall back to boot1 prompt.
           ZFS pool in Disk2 is shown before one in Disk1.

Set 3: Disk2 is removed. (Disk1 only environment)

  3-1) ZFS only has /boot/loader.efi of head.
        -> stable/10 in Disk1 ZFS boots fine.

  3-2) Same as 2-2 without Disk2.
        -> Fail to boot. Fall back to loader prompt.
           (Of course. Specified root device doesn't exists.)

  3-3) Same as 2-4 without Disk2.
        -> stable/10 in Disk1 ZFS boots fine.

  3-4) Both UFS and ZFS have /boot/loader.efi of head.
        -> stable/10 in Disk1 ZFS boots fine.


Regards.


On Thu, 10 Dec 2015 12:41:15 +0000
Steven Hartland <steven at multiplay.co.uk> wrote:

> Ive literally just got this working on 10.2 after working on the code
> posted on the review which you can find here:
> https://reviews.freebsd.org/D4104
> 
> If you're happy running current then the patch file I linked in my comment
> should apply cleanly and just work.
> 
> If you want 10.x then there's quite a bit more needed. As I said I do have
> this working so can post patches when I'm back in the office.
> 
> Either way once applied a standard efi install just works. Essentially
> create efi partition and use gpart to install the efi bootcode and away you
> go.
> 
> I've just used this with a custom mfsbsd iso to perform and 10.2-RELEASE
> ZFS boot install on some Intel nvme disks setup as raidz2, which only
> support efi boot.
> 
> On 10 Dec 2015, at 12:18, krad <kraduk at gmail.com> wrote:
> 
> Hi, I need to get one of my machines converted over from bios GPT zfsroot
> boot to efi. I know you can boot freebsd under EFI with a ufs kernel but
> this isnt the route i want. There are patches under test for EFI zfs root.
> However when I read the thread it was unclear which version of these
> patches were needed and where to get them. Does anyone know where they are,
> if there are any prebuilt zfsloader etc binaries, or if the patches have
> made it to head yet?
> 
> Also does anyone have any pointers or good experience with grub efi and zfs
> on root? I'm considering this option as it would make booting into specific
> boot environments easier
> _______________________________________________
> freebsd-current at freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscribe at freebsd.org"
> _______________________________________________
> freebsd-current at freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscribe at freebsd.org"
> 


-- 
Tomoaki AOKI    junchoon at dec.sakura.ne.jp