Encrypted (GELI) root on ZFS troubles

[Andriy Gapon]

On 02/10/2014 00:27, Karl Denninger wrote:
> So here's the fun part of what I'm trying to do (and getting frustrated
> with)
> 
> I have set up a GPT disk with the following setup:
> 
> =>       34  625142381  da2  GPT  (298G)
>          34          6       - free -  (3.0K)
>          40       1024    1  freebsd-boot  (512K)
>        1064    4194304    2  freebsd-zfs  [bootme]  (2.0G)
>     4195368  134217728    3  freebsd-swap  (64G)
>   138413096  486729312    4  freebsd-zfs  (232G)
>   625142408          7       - free -  (3.5K)
> 
> Then on freebsd-boot I have written the bootloaders.
> 
> The "bootme" filesystem has *only* the /boot directory copied over from
> the rest of the system's root directory (that is, the kernel, loadables,
> /boot/loader.conf, etc); that pool is called "zboot"
> 
> Partition 4 has the label "root0" on it, and thus shows up in /dev/gpt. 
> I have initialized that with geli, set the boot option flag (that is,
> prompt on boot) and created a pool called "root" on the resulting .eli
> device and then put the system on that.  That's all ok.
> 
> Finally, I set the bootfs on that latter pool.  There is no bootfs set
> on /zboot:
> 
> # zpool get bootfs zboot
> NAME   PROPERTY  VALUE   SOURCE
> zboot  bootfs    -       default
> 
> It is set on the root pool to the proper filesystem:
> 
> # zpool get bootfs root
> NAME  PROPERTY  VALUE              SOURCE
> root  bootfs    root/R/10.1-CLEAN  local
> 
> The problem is that when the system boots geli "finds" the raw device
> (in this case /dev/da0p4), prompts for the password and attaches there
> instead of in /dev/gpt.  The gpt label is missing --- and equally bad
> the "root" pool does not appear to import at boot time either.
> 
> As a result the system tries to mount root from /zboot (even though it's
> not been told to, and HAS been told where to mount off the root pool),

As far as *I* can see, you have not told the kernel what your root fs should be,
so it is using a default root filesystem which the same filesystem from where
the kernel itself was loaded.

> but there's no init in there (or anything else other than the boot
> filesystem itself) and as a result I get an immediate panic.

-- 
Andriy Gapon