ZFS root set to readonly=on temporary at boot

[Xīcò]

On Thu, Feb 19, 2015 at 12:12:54PM +0000, krad wrote:
> Check your bootfs and the / file system actually match up. Its quite easy
> to get odd things happening if you have the bootfs set to a filesystem that
> then has an fstab which then tells / is on another fs. Also check the
> loader.conf on the bootfs. In my experiance its safer to have clean fstabs,
> and nothing in the loader.conf (in relation to this) and just relay on the
> bootfs pool property

Thanks for the tips. I still could not figure it out though. My
partition scheme is as follows:

  # gpart show
  =>        34  3907029101  ada0  GPT  (1.8T)
            34         128     1  freebsd-boot  (64K)
           162     8388608     2  freebsd-swap  (4.0G)
       8388770  3898640365     3  freebsd-zfs  (1.8T)

with a bootcode installed by:

  # gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada0

The bootfs property of the zpool seems to be set to the correct root:

  # zpool get bootfs zroot
  NAME   PROPERTY  VALUE   SOURCE
  zroot  bootfs    zroot   local
  # zfs list zroot
  NAME    USED  AVAIL  REFER  MOUNTPOINT
  zroot  2.16G  1.75T   509M  /

The /etc/fstab only lists the swap partition (and there are no other
fstab files, nor any unmounted snapshot/partitions):

  # cat /etc/fstab
  /dev/label/swap none swap sw 0 0

My loader.conf appears to be required for the server to boot:

  # cat /boot/loader.conf 
  zfs_load="YES"
  vfs.root.mountfrom="zfs:zroot"

Also, the root is mounted in read/write when manually imported from
another system:

  # zpool import -R /mnt zroot
  # zfs get readonly zroot
  NAME   PROPERTY  VALUE   SOURCE
  zroot  readonly  off     temporary

Anyway, it is not that huge a problem, but it is quite inconvenient not
to understand what could be the reason behind that. As far as I can
tell, it may be related to the fact that my root was not created in its
own dataset:

  # zfs list
  NAME                    USED  AVAIL  REFER  MOUNTPOINT
  zroot                  2.16G  1.75T   509M  /
  zroot/ezjail            984M  1.75T    27K  /usr/jails
  zroot/var               284M  1.75T  10.6M  /var
  […]

I might check that later on.

Best,

Xīcò