Encrypted (GELI) root on ZFS troubles

[Andriy Gapon]

On 02/10/2014 15:52, Karl Denninger wrote:
> 
> On 10/2/2014 3:45 AM, Andriy Gapon wrote:
>> On 02/10/2014 07:50, Karl Denninger wrote:
>>> 1. Having the kernel able to cross pools and examine the bootfs property
>>> would be fairly useful.
>> I don't understand your suggestion.  Every pool has bootfs property (if it's not
>> explicitly set then that means that it just has its default value).  So which of
>> the bootfs properties must be honored and why?  At present we honor bootfs of
>> the "current" pool, which is the only distinguished pool.
> Except default is "from whence one came" and yes, I understand the
> problem if there's a conflict.  I would argue that if the parameter is
> set on the pool the kernel booted from it should be honored, but if it
> is NOT set (that is, if it is defaulted) then if it is validly set on a
> different pool that one should be honored.

As I've said earlier, whether bootfs is set on a pool or not does mean anything
in terms of pool's 'bootability'.  Unset bootfs is equivalent to bootfs being
set to $poolname.  That's it, there are no side-effects, no hidden meaning.

> We already do this in that if /boot/loader.conf is set that's honored,
> and this sounds inviolate but it in fact is not for the below reason.
> 
> I do understand the problem in the general sense with "well what happens
> if there are two or more bootfs parameters set on other pools but none
> on the pool the kernel loaded from" but here's the thing -- that risk is
> already there, in that there's already all sorts of "fun" possible with
> adapters that don't enumerate reliably in order, especially if the
> intended boot device goes offline.  My LSI adapters, in particular
> (which are a great example because lots of people use these with SAS
> expanders and lots of plugged in drives, myself included, as they work
> well and are pretty fast), will happily look for another bootable device
> that happens to be plugged in if your primary(s) are offline and, if you
> have one in the machine, it will attempt to boot it.  Sucks to be you if
> that turns out to be a bootable environment.
> 
> So what I think makes sense is the following:
> 
> 1. If /boot/loader.conf has vfs.root.mountfrom set, honor it (yes, there
> is a risk to this too; you might have booted from an unintended disk!)
> 
> 2. If we booted from zfs and bootfs is set non-default on the pool you
> booted from then honor it, otherwise honor the first non-default bootfs
> setting you find on any pool that is mounted during kernel init (which
> for a geli-encrypted pool means one that had the "-b" flag set on its
> members, otherwise it's not available until after init starts.)  I
> understand this has risks but so does an adapter deciding to boot from
> the "wrong" disk; there's no reason for bootfs to be set non-default on
> a pool that isn't used for booting.
> 
> 3. If neither (1) or (2) is true then mount root from the pool/device
> you loaded the kernel from.
> 
> I can certainly understand that not being the default (you could get
> reamed pretty good if you booted a non-corresponding kernel .vs. the
> rest of the environment) but IMHO it violated the principle of least
> astonishment when I went to set this up and the implication that bootfs
> on *a* pool was sufficient as a hint to the kernel.
> 


-- 
Andriy Gapon