[Bug 206521] Can't decrypt disks on ZFS+Geli installation after order of devices changed

bugzilla-noreply at freebsd.org bugzilla-noreply at freebsd.org
Sat Jan 23 13:54:48 UTC 2016


https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=206521

            Bug ID: 206521
           Summary: Can't decrypt disks on ZFS+Geli installation after
                    order of devices changed
           Product: Base System
           Version: 10.2-RELEASE
          Hardware: Any
                OS: Any
            Status: New
          Severity: Affects Only Me
          Priority: ---
         Component: bin
          Assignee: freebsd-bugs at FreeBSD.org
          Reporter: florian.ermisch at alumni.tu-berlin.de

(Rendered Markdown here: https://gist.github.com/0xf10e/ddefc6fad77d6b51672f)

## Preface

I added a 256GB mSATA SSD to my Lenovo x220
of which I planned to use a nice chunk as L2ARC
for the SATA HDD. Even without a SATA disk 
present the SSD showed up as HDD2 in the BIOS.

Made a fresh installation of FreeBSD 10.2 amd64
with the memstick image and chose ZFS+Geli. 
Booted, worked, everything just fine. I added a 
SATA HDD which the BIOS listed as HDD0 just as 
I expected from the previous SATA-only setup.
When I now tried to boot from the SSD I was 
prompted for the GELI key of a different device
than before, `ada1p3` instead of `ada0p3`.
But my passphrase wasn't accepted. Never.
Re-installed, added HDD, same result. 
Reinstalled with HDD present, the removed the 
HDD, same result (then prompted for `ada0p3` 
instead of `ada1p3`, of course).

When I restored the configuration which
was present during the installation process
(either remove the HDD or add it back in) 
everything worked fine again.

## Using bhyve to reproduce

Created a VM, added more disks:
```
floh at fuchi-cyber220:~:1305% iohyve info
Name           Size  RAM  CPU  OS       Loader
fbsd102        -     2G   1    default  bhyveload
fbsd102/disk0  2G    2G   1    default  bhyveload
fbsd102/disk1  2G    2G   1    default  bhyveload
fbsd102/disk2  4G    2G   1    default  bhyveload
```

Install FreeBSD 10.2 (the one I used for my laptop):

  ```
  floh at fuchi-cyber220:~:1305% sudo iohyve install fbsd102
FreeBSD-10.2-RELEASE-amd64-disc1.iso
  Installing fbsd102...
  floh at fuchi-cyber220:~:1307% sudo iohyve console fbsd102
  Starting console on fbsd102...
  ~~. to escape console [uses cu(1) for console]
  Connected
  ```

* Chose Auto (ZFS), added only ada2/disk2 to the rootpool.

```
┌────────────────ZFS Configuration───────────────────┐
│ Configure Options:                                 │
│ ┌────────────────────────────────────────────────┐ │
│ │ >>> Install          Proceed with Installation │ │
│ │ T Pool Type/Disks:   stripe: 1 disk            │ │
│ │ - Rescan Devices     *                         │ │
│ │ - Disk Info          *                         │ │
│ │ N Pool Name          zroot                     │ │
│ │ 4 Force 4K Sectors?  YES                       │ │
│ │ E Encrypt Disks?     YES                       │ │
│ │ P Partition Scheme   GPT                       │ │
│ │ S Swap Size          256m                      │ │
│ │ M Mirror Swap?       NO                        │ │
│ │ W Encrypt Swap?      YES                       │ │
│ └────────────────────────────────────────────────┘ │
├────────────────────────────────────────────────────┤
│             <Select>       <Cancel>                │
└────────────────────────────────────────────────────┘
```

* proceed with installation.
* remove disk1 from the VM:
  ```
  floh at fuchi-cyber220:~:1313% sudo iohyve remove fbsd102 disk1
  Are you sure you want to remove disk1 from fbsd102 [Y/N]? y
  floh at fuchi-cyber220:~:1314% iohyve info
  Name           Size  RAM  CPU  OS       Loader
  fbsd102        -     2G   1    default  bhyveload
  fbsd102/disk0  2G    2G   1    default  bhyveload
  fbsd102/disk2  4G    2G   1    default  bhyveload
  ```

* shuffle disks around:
  ```
  floh at fuchi-cyber220:~:1340% sudo zfs rename zroot/iohyve/fbsd102/disk{0,1}
  floh at fuchi-cyber220:~:1342% sudo zfs rename zroot/iohyve/fbsd102/disk{2,0}
  floh at fuchi-cyber220:~:1342% sudo zfs rename zroot/iohyve/fbsd102/disk{1,2}
  ```
* now the 4GB one is `disk0`:
  ```
  floh at fuchi-cyber220:~:1343% iohyve info
  Name           Size  RAM  CPU  OS       Loader
  fbsd102        -     2G   1    default  bhyveload
  fbsd102/disk0  4G    2G   1    default  bhyveload
  fbsd102/disk2  2G    2G   1    default  bhyveload
  ```

* boot the VM and try to get the rootpool imported:
```
floh at fuchi-cyber220:~:1344% sudo iohyve start fbsd102
Starting fbsd102... (Takes 15 seconds for FreeBSD guests)
floh at fuchi-cyber220:~:1344%
floh at fuchi-cyber220:~:1345% sudo iohyve console fbsd102
Starting console on fbsd102...
~~. to escape console [uses cu(1) for console]
Connected
  /
  ______               ____   _____ _____
 |  ____|             |  _ \ / ____|  __ \
 | |___ _ __ ___  ___ | |_) | (___ | |  | |
 |  ___| '__/ _ \/ _ \|  _ < \___ \| |  | |
 | |   | | |  __/  __/| |_) |____) | |__| |
 | |   | | |    |    ||     |      |      |
 |_|   |_|  \___|\___||____/|_____/|_____/    ```                        `
                                             s` `.....---.......--.```   -/
 +============Welcome to FreeBSD===========+ +o   .--`         /y:`      +.
 |                                         |  yo`:.            :o      `+-
 |  1. Boot Multi User [Enter]             |   y/               -/`   -o/
 |  2. Boot [S]ingle User                  |  .-                  ::/sy+:.
 |  3. [Esc]ape to loader prompt           |  /                     `--  /
 |  4. Reboot                              | `:                          :`
 |                                         | `:                          :`
 |  Options:                               |  /                          /
 |  5. [K]ernel: kernel (1 of 2)           |  .-                        -.
 |  6. Configure Boot [O]ptions...         |   --                      -.
 |                                         |    `:`                  `:`
 |                                         |      .--             `--.
 |                                         |         .---.....----.
 +=========================================+


/boot/kernel/kernel text=0xfc8de8 data=0x1283b0+0x207880
syms=[0x8+0x145350+0x8+0x15fe20]
/boot/kernel/zfs.ko size 0x2f9b00 at 0x199e000
loading required module 'opensolaris'
/boot/kernel/opensolaris.ko size 0x6048 at 0x1c98000
/boot/kernel/geom_eli.ko size 0x21568 at 0x1c9f000
loading required module 'crypto'
/boot/kernel/crypto.ko size 0x35318 at 0x1cc1000
/boot/encryption.key size=0x1000
/boot/kernel/aesni.ko size 0x5a30 at 0x1cf8000
/boot/zfs/zpool.cache size=0x8bc
Booting...
Copyright (c) 1992-2015 The FreeBSD Project.

[... lots kernel messages ...]

ada1: <BHYVE SATA DISK 001> ACS-2 ATA SATA 3.x device
ada1: Serial Number BHYVE-D12E-A75C-27F5
ada1: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes)
ada1: Command Queueing enabled
ada1: 2048MB (4194304 512 byte sectors: 16H 63S/T 4161C)
ada1: Previously was known as ad6
random: unblocking device.
Timecounter "TSC-low" frequency 1345403547 Hz quality 1000
Enter passphrase for ada0p4:
GEOM_ELI: Wrong key for ada0p4. Tries left: 2.
Enter passphrase for ada0p4:
GEOM_ELI: Wrong key for ada0p4. Tries left: 1.
Enter passphrase for ada0p4:
GEOM_ELI: Wrong key for ada0p4. No tries left.
Enter passphrase for diskid/DISK-BHYVE-F485-E5AF-7C59p4:
GEOM_ELI: Wrong key for diskid/DISK-BHYVE-F485-E5AF-7C59p4. Tries left: 2.
Enter passphrase for diskid/DISK-BHYVE-F485-E5AF-7C59p4:
GEOM_ELI: Wrong key for diskid/DISK-BHYVE-F485-E5AF-7C59p4. Tries left: 1.
Enter passphrase for diskid/DISK-BHYVE-F485-E5AF-7C59p4:
GEOM_ELI: Wrong key for diskid/DISK-BHYVE-F485-E5AF-7C59p4. No tries left.
Enter passphrase for gpt/zfs0:
GEOM_ELI: Wrong key for gpt/zfs0. Tries left: 2.
Enter passphrase for gpt/zfs0:
GEOM_ELI: Wrong key for gpt/zfs0. Tries left: 1.
Enter passphrase for gpt/zfs0:
GEOM_ELI: Wrong key for gpt/zfs0. No tries left.
Trying to mount root from zfs:zroot/ROOT/default []...
Mounting from zfs:zroot/ROOT/default failed with error 2.

Loader variables:
  vfs.root.mountfrom=zfs:zroot/ROOT/default

Manual root filesystem specification:
  <fstype>:<device> [options]
      Mount <device> using filesystem <fstype>
      and with the specified (optional) option list.

    eg. ufs:/dev/da0s1a
        zfs:tank
        cd9660:/dev/acd0 ro
          (which is equivalent to: mount -t cd9660 -o ro /dev/acd0 /)

  ?               List valid disk boot devices
  .               Yield 1 second (for background tasks)
  <empty line>    Abort manual input

mountroot>
panic: mountroot: unable to (re-)mount root.
cpuid = 0
KDB: stack backtrace:
#0 0xffffffff80984e30 at kdb_backtrace+0x60
#1 0xffffffff809489e6 at vpanic+0x126
#2 0xffffffff809488b3 at panic+0x43
#3 0xffffffff809e7f5f at vfs_mountroot+0x1eaf
#4 0xffffffff808f03b3 at start_init+0x53
#5 0xffffffff8091244a at fork_exit+0x9a
#6 0xffffffff80d30d2e at fork_trampoline+0xe
Uptime: 4m16s
```

And, no, srsly, I don't need more than 9 tries to type "foobar".

Changing the order of disks back would fix the problem.
As there's no BIOS menu to choose the boot device in bhyve
having no bootcode in (the original) disk0 makes this a
tiny bit difficult with bhyve.

-- 
You are receiving this mail because:
You are the assignee for the bug.


More information about the freebsd-bugs mailing list