RE: From UFS to ZFS as root on RBPI4

> From: John Kennedy
> 
> I'd probably recommend doing a bsdinstall onto the
> USB-attached SDcard, but the RootOnZFS page is what I used back when that
> was a new thing.

I had a ZFS-on-SSD RPi4 go south after rebuilding and installing my custom kernel.  On reboot it would either fail to find the pool, or (apparently) permanently stall during boot.  I gave up and decided to reinstall, so booted to an SD card burned from https://download.freebsd.org/releases/arm64/aarch64/ISO-IMAGES/13.1/FreeBSD-13.1-RELEASE-arm64-aarch64-RPI.img.xz, ran "bsdinstall", and installed on the SSD.

That left me with an unbootable system, as it does not install any RPi firmware.

After farting around trying to get the right files from rpi-firmware installed in the right place, I eventually gave up and wiped out /boot on the SSD and copied everything from the card's /boot.  Note that BSD install creates a /boot/efi directory for the UEFI partition, with only two files (in subdirectories), whereas the SD card has /boot/MSDOS with a boatload of files, and different subdirectories, so I had to adjust the mount point in /etc/fstab for that.  My RPi is now (again) booting to the SSD (though I'll need to clone my customized kernel from git and (slowly) rebuild it).

Also note that I had previously used the Bootloader "USB Boot" image from Pi Imager to set up the RPi4 to boot from USB instead of SD card.

It might be possible to drop to a shell from inside "bsdinstall" to get the RPi firmware in place before exiting the installer, but I don't know what actions would be needed for that.  Presumably, "pkg install rpi-firmware" followed by copying file from /usr/local/share/rpi-firmware to somewhere...  And maybe "pkg install u-boot-rpi4" and copying something from there as well.