Possible (or smart) to put freebsd-boot on USB stick for root-on-ZFS?

[Warren Block]

On Tue, 24 Mar 2015, Jason Birch wrote:

> Hey there,
>
> I'm looking at a relatively old resource
> (https://wiki.freebsd.org/RootOnZFS/GPTZFSBoot/9.0-RELEASE) about how
> to run root on ZFS for 9.0, and I noticed the section on installing
> the boot section to all drives that make up the root. In my setup,
> I'll only be mirroring two SSDs on 10.1, but it made me consider the
> possibility of having the freebsd-boot partition on a USB stick rather
> than on each drive itself for basically the following reason:
>
> Should an SSD die, I'd need to say "boot from this other device" to
> get my system back up and running, do the original partitioning magic
> on the replacement device, get that back into the vdev... Should the
> USB stick die, I'd need to simply replace it with one that had the
> same image (that is, only the boot partition), and `zpool replace` a
> blank device (I won't have a swap partition).
>
> However, I can think of some downsides as well - namely that the USB
> stick is probably more likely to die than the SSDs, and that the image
> on the USB stick will change over time (This gets mounted as /boot? or
> am I mistaken here... This would plague the freebsd-boot on the SSDs
> as well, with drift...)

The freebsd-boot partition does not have a filesystem.  The only thing 
in it is the binary bootcode, like gptzfsboot(8).

> Am I misinterpreting the point of freebsd-boot? Does /boot actually
> end up living on the ZFS mount, and freebsd-boot just contains enough
> information to read the kernel and other goodies required to bring up
> a full system from a ZFS dataset?

That is correct.

> Is my thought to use a USB stick for this partition a little thick or 
> actually worth trying out?

The bootcode changes rarely, and is usually not a problem.  Having a 
separate, duplicate /boot is sometimes done for geli(8).  It's kind of a 
pain, because changes must be made in two places.  For a mirror with the 
OS on it, I'd put a freebsd-boot partition on each drive, manually 
install the bootcode to each, and call it good.