Encrypted ZFS boot failure on 11.*

[Gary Corcoran]

The below all refer to AMD64 versions.

I wanted to create a backup server from a few years old machine, so I bought
five drives to create a ZFS raidz1.  And naturally wanted to put the latest
non-experimental version of FreeBSD on it, 11.*.  And I wanted the drives to
be encrypted.

But every version of FreeBSD 11 I tried, including the latest 11.1-BETA1, has
the same bug.  Using the "guided Auto ZFS" option in the installer, I chose
raidz1, GPT (BIOS) partioning, and encryption on.  Installations would proceed
without any problem.  But then, upon rebooting into the newly installed system,
I would get the following error messages (after entering the GELI passphrase):

error 1 lba 90728
failed to clear pad 2 area of primary vdev
failed to read pad 2 area of primary vdev
ZFS: i/o error - all block copies unavailable

I was worried that I might have gotten a bad disk drive.  But apparently no.
For when I did the exact same installation, except no encryption, there were
no errors and it successfully booted.  From a little bit of googling I understand
that the pad2 area has to do with a one-time "nextboot" ZFS feature, and thus
the area is cleared after booting.  I presume that is true whether or not the
disks are encrypted, so having a successful boot without encryption should
mean that the pad2 area of the disk(s) was accessed without a problem, as long
as those sectors aren't encrypted.

Next I tried installing FreeBSD 10.3-Release.  Doing the same type of installation,
with encryption ON, the system installed and booted without any problems.

So in summary, encrypted ZFS works in version 10.3, but is broken in all 11.* versions.

I am very surprised that no one else has run into this major problem.

I've done so many installs that I am tired of doing them, and since the 10.3 install
works, I'm just going to use that.  But I thought that, especially since you're in
BETA, this should be reported.

Gary

Retired Software Engineer
FreeBSD user since 2.2