Encrypted (GELI) root on ZFS troubles
dweimer
dweimer at dweimer.net
Thu Oct 2 01:15:45 UTC 2014
On 10/01/2014 4:27 pm, Karl Denninger wrote:
> So here's the fun part of what I'm trying to do (and getting frustrated
> with)
>
> I have set up a GPT disk with the following setup:
>
> => 34 625142381 da2 GPT (298G)
> 34 6 - free - (3.0K)
> 40 1024 1 freebsd-boot (512K)
> 1064 4194304 2 freebsd-zfs [bootme] (2.0G)
> 4195368 134217728 3 freebsd-swap (64G)
> 138413096 486729312 4 freebsd-zfs (232G)
> 625142408 7 - free - (3.5K)
>
> Then on freebsd-boot I have written the bootloaders.
>
> The "bootme" filesystem has *only* the /boot directory copied over from
> the rest of the system's root directory (that is, the kernel,
> loadables,
> /boot/loader.conf, etc); that pool is called "zboot"
>
> Partition 4 has the label "root0" on it, and thus shows up in /dev/gpt.
> I have initialized that with geli, set the boot option flag (that is,
> prompt on boot) and created a pool called "root" on the resulting .eli
> device and then put the system on that. That's all ok.
>
> Finally, I set the bootfs on that latter pool. There is no bootfs set
> on /zboot:
>
> # zpool get bootfs zboot
> NAME PROPERTY VALUE SOURCE
> zboot bootfs - default
>
> It is set on the root pool to the proper filesystem:
>
> # zpool get bootfs root
> NAME PROPERTY VALUE SOURCE
> root bootfs root/R/10.1-CLEAN local
>
> The problem is that when the system boots geli "finds" the raw device
> (in this case /dev/da0p4), prompts for the password and attaches there
> instead of in /dev/gpt. The gpt label is missing --- and equally bad
> the "root" pool does not appear to import at boot time either.
>
> As a result the system tries to mount root from /zboot (even though
> it's
> not been told to, and HAS been told where to mount off the root pool),
> but there's no init in there (or anything else other than the boot
> filesystem itself) and as a result I get an immediate panic.
>
> If I boot off a different (working) zfs-based system the probe still
> finds the "prompt during boot" flag on that gpt partition and asks for
> the password on the device. I can see the pool; zpool import shows it:
>
> pool: root
> id: 17719633931604198170
> state: ONLINE
> action: The pool can be imported using its name or numeric identifier.
> config:
>
> root ONLINE
> da2p4.eli ONLINE
>
> Not so good.
>
> If I detach that the device reappears in /dev/gpt; I can then attach
> geli and import the pool in either location. Putting the cache file
> from the previous imported state in the zboot/boot/zfs directory
> doesn't
> help (nor does removing the cache file entirely)
>
> More-interestingly if I reboot the cloned system with the root pool
> imported it does come back up, even though the device is the base
> (da2p4.eli) rather than in the /dev/gpt directory.
>
> Anyone know what's going on here? And is there a way to have geli
> attach during boot-time off the /dev/gpt directory instead of on the
> base device partition name?
On my work laptop (not turned on so I am going by memory on this), I
have a similar setup using a USB thumb drive for the boot volume. My
setup is as follows and works quite well, perhaps this will help you.
Thumb Drive
da0
Disk Drive
ada0
da0 has a GPT table of
da0 GPT (8G)
1 freebsd-boot (512k) -- /dev/gpt/usbboot
2 freebsd-zfs (8G) -- /dev/gpt/usbzfs
gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da0
ada0 has a GPT table of
1 freebsd-swap (8G) -- /dev/gpt/swap
2 freebsd-zfs (222G) -- /dev/gpt/zroot
I used geli init -b /dev/gpt/zroot
when attached /dev/gpt/zroot.eli
swap is auto encrypted at boot using the fstab line
/dev/gpt/swap.eli none none swap sw 0 0
I believe they devices only show up as /dev/gpt/... if the -l ... option
is used to set a label on the partition at creation time.
2 configured zpools
usbzfs
gpt/usbzfs
zroot
gpt/zroot.eli
zpool set bootfs=usbzfs/boot usbzfs
zpool set bootfs=zroot/ROOT/installation zroot (not sure if this does
anything, I just set it)
usbzfs/boot has a mountpoint of /zfsboot
loader.conf:
zfs_load="YES"
vfs.root.mountfrom="zfs:zroot/ROOT/install"
copied /boot to /zfsboot/boot
zpool export usbzfs
It will still boot after the zpool has been exported if the devices is
found, just doesn't get mounted, in my case this means I can remove the
USB thumb drive as soon as root is remounted from the geli partition,
after entering the password without causing any issues.
I can send you the full gpt output and zpool status information tomorrow
morning when I am back in the office on my laptop if you still need help
getting yours working.
--
Thanks,
Dean E. Weimer
http://www.dweimer.net/
More information about the freebsd-stable
mailing list