Difficulties to use ZFS root: ROOT MOUNT ERROR

[Martin Matuska]

On 5. 9. 2011 21:54, Jeremie Le Hen wrote:
> Hi list,
>
> I've followed the instructions documented here:
>     http://wiki.freebsd.org/RootOnZFS/ZFSBootPartition
>
> The kernel starts correctly so this rules out any problem regarding
> boot0, zfsboot and ZFS loader.
>
> But when the kernel tries to mount the root filesystem, it fails with
> the following output:
>
> % Trying to mount root from zfs:zroot
> % ROOT MOUNT ERROR:
> % If you have invalid mount options, reboot, and first try the following from
> % the loader prompt:
> % 
> %      set vfs.root.mountfrom.options=rw
> % 
> % and then remove the invalid mount options from /etc/fstab.
> % 
> % Loader variables:
> % vfs.root.mountfrom=zfs:zroot
> % vfs.root.mountfrom.options=rw
>
> >From a netboot'd FreeBSD:
>
>  # zfs import zroot
> % # zpool get bootfs zroot
> % NAME   PROPERTY  VALUE   SOURCE
> % zroot  bootfs    zroot   local
> % 
> % # zfs list -o name,canmount,mountpoint
> % NAME                       CANMOUNT  MOUNTPOINT
> % zroot                            on  legacy
> % zroot/tmp                        on  /tmp
> % zroot/usr                        on  /usr
> % zroot/usr/home                   on  /usr/home
> % zroot/usr/ports                  on  /usr/ports
> % zroot/usr/ports/distfiles        on  /usr/ports/distfiles
> % zroot/usr/ports/packages         on  /usr/ports/packages
> % zroot/usr/src                    on  /usr/src
> % zroot/usr/src8                   on  /usr/src8
> % zroot/var                        on  /var
> % zroot/var/crash                  on  /var/crash
> % zroot/var/db                     on  /var/db
> % zroot/var/db/pkg                 on  /var/db/pkg
> % zroot/var/empty                  on  /var/empty
> % zroot/var/log                    on  /var/log
> % zroot/var/mail                   on  /var/mail
> % zroot/var/run                    on  /var/run
> % zroot/var/tmp                    on  /var/tmp
> % # zfs export zroot
>
> /boot/zfs/zpool.cache exists in the zroot filesystem:
>
> % # zpool import -R /mnt zroot
> % # zfs set mountpoint=/ zroot
> % # ls -l /mnt/boot/zfs/zpool.cache
> % -rw-r--r--  1 root  wheel  924 Sep  5 07:31 /mnt/boot/zfs/zpool.cache
> % # grep zfs /mnt/boot/loader.conf /mnt/etc/rc.conf 
> % /mnt/boot/loader.conf:zfs_load="YES"
> % /mnt/boot/loader.conf:vfs.root.mountfrom="zfs:zroot"
> % /mnt/etc/rc.conf:zfs_enable="YES"
>
>
> Any idea why this error occurs?
>
> Thanks
It might be a problem in the zpool.cache.
If you read the zpool(8) manpage properly, you will find this:

-R root
  Equivalent to "-o cachefile=none,altroot=root"

If you mount a pool with an alternate root and want to update the
cachefile, you have to explicitly state the cachefile.
(e.g. zpool import -o altroot=/mnt -o cachefile=/tmp/zpool.cache zroot)
Second, you should not have an exported pool for booting (from viewpoint
of target system's zfs.cache).
Third, you don't need a legacy mount for zroot. You can leave it to "/"
but you don't have to.
I personally prefer having everyting one level deeper (e.g. pool/root,
pool/root/var, etc.).

Therefore I suggest:
zpool import -o altroot=/mnt -o cachefile=/tmp/zpool.cache zroot
zfs set mountpoint=/ zroot
cp /tmp/zpool.cache /mnt/boot/zfs/zpool.cache
shutdown -r now
(as you can see I have not exported the pool)

P.S: mfsBSD is your friend here - http://mfsbsd.vx.sk

-- 
Martin Matuska
FreeBSD committer
http://blog.vx.sk