ZFS Root Won't Mount - Unknown Filesystem

[Matt Mullins]

On Sat, Jan 7, 2012 at 7:44 AM, Drew Tomlinson <drew at mykitchentable.net> wrote:
> I'm attempting a new install of 9.0-RC3 amd64.  My system has 4 500 GB
> drives.  Using this tutorial as a guide:
>
> http://www.aisecure.net/2011/11/28/root-zfs-freebsd9/

When I built my ZFS-root system, I did most of these things, except I
had a slightly different setup for the root filesystem itself.  What I
did is akin to:
  # zpool import -o altroot=/target -o cachefile=/tmp/zpool.cache zroot
  # zfs set mountpoint=/ zroot
  # cp -a /tmp/zpool.cache /target/boot/zfs/zpool.cache

I did not manually set mountpoints for anything but the root
filesystem.  It sounds like manually-set mountpoints are the way most
people do it, but you can try to clear the mountpoints by doing:
  # zfs inherit mountpoint zroot/fs
for each filesystem in your root pool.

> I created a boot and a freebsd-zfs partition on each drive.  Then I created
> a raid1z pool using all 4 drives.  I followed the rest of the tutorial
> exactly and ensured that I copied the zpool.cache to boot/zfs.
>
> When I try to boot my new system, it all goes fine up until it's time to
> mount zfs:zroot.  It fails with an "error 2" "unknown filesystem" error.  I
> don't know if this means anything but at the mountfrom prompt, the system
> will not accept any keyboard input.  Same keyboard works fine when booted
> into LiveCD.

I had that same problem with mine for a while, and it turned out that
importing with the "altroot" option implies "cachefile=none"; until I
realized I needed to also specify "cachefile=/some/path", I had
accidentally ended up with a /boot/zpool.cache that didn't actually
reference any zpools.

> Unfortunately because I can't figure out how to get a LiveCD type
> environment with sshd running, I can't copy and paste exact error messages
> or command outputs.

I was using PXE/NFS booting to install this machine, so unfortunately
I can't help you here.

> I've searched and the two things that seem to be important are that there's
> a zpool.cache file and that the zfs partitions are correct.  A 'gpart show
> -l' shows my partitions something close to this:
>
> 34 <big number>    ada0    GPT (456G)
> 34    128             1       null (128K)
> 162 <big number>   2       disk0 (456G)
>
> What have I done wrong and what do I need to do to get my zfs:zroot pool
> mounted as root?

It sounds like you're almost there!  My guess is that the cache file
is what is missing/incorrect.

Reading over some man pages, make sure you don't do a "zpool export"
before you copy the cache file; exporting the array removes it from
the cache and/or deletes the cache file entirely.

If you end up with a LiveCD that lets you copy these things, it might
help to see
  # zpool list -o name,altroot,cachefile
  # zpool status
  # zfs list -o name,mountpoint,mounted

Hope some of this helps.
--
Matt Mullins