ZFS is auto-mounting in wrong order

Paul Mather paul at gromit.dlib.vt.edu
Fri Apr 19 14:31:34 UTC 2019

Jim Long <freebsd-questions at umpquanet.com> wrote:

> I have a system that boots from ZFS.  I recently upgraded it from
> FreeBSD 11.2-STABLE to 12.0-STABLE:
> FreeBSD electron 12.0-STABLE FreeBSD 12.0-STABLE r346293 GENERIC  amd64
> I'm seeing certain directories as empty, when they should have lots of
> content.  The problem can be worked around by unmounting key
> filesystems, and re-mounting them in the proper order.
> Troubleshooting leads me to believe that the ZFS system is not mounting
> them in the proper order, but I don't know of any way to either log the
> order that they are mounted in at boot time, nor to inspect on a
> hypothetical "dry-run" basis what ZFS thinks the correct mount order is.
> I did not have this problem prior to the 11 -> 12 upgrade, if that
> matters.

I encountered this problem in upgrading from 11-STABLE -> 12-STABLE.  Under  
11-STABLE and prior, when upgrading in single-user mode I would use "zfs  
mount -a" to mount my ZFS file systems.  Under 12-STABLE, this resulted in  
many "empty directories" as you mention.  It caused the subsequent "make  
installworld" to fail.

In my case, I discovered it was the way my boot environment had been set up  
that was causing this.  The file systems under my boot environment "root"  
(pool/ROOT) were all marked as "noauto".  Under 11-STABLE, "zfs mount -a"  
would successfully mount these file systems (even though the man page said  
it shouldn't).  Under 12-STABLE, "zfs mount -a" acts in accordance with the  
man page and does not mount these file systems in response to "zfs mount  

My workaround was now to mount ZFS file systems as the system does: first  
execute /etc/rc.d/zfsbe and then /etc/rc.d/zfs, i.e., "/etc/rc.d/zfsbe  
start ; /etc/rc.d/zfs start".  That correctly mounts the active boot  
environment and then the rest of the ZFS file systems on top of it.

I notice you have an "electron/system/ROOT" ZFS file system, so maybe you  
are using boot environments, too?  Do you have any ZFS file systems marked  
"noauto" for the "canmount" property?  It doesn't seem so, but I thought I  
would double-check.

> This seems like strong evidence that /data/backup is getting mounted
> before /data gets mounted, and ZFS is auto-creating the mountpoint
> directory.
> How can I definitively SEE the order in which ZFS will mount
> filesystems, either in a "dry-run" mode, or have it logged in real time
> during the boot process?

The "service -e" command will show you the order in which services will be  
started.  In my 12-STABLE system, ZFS related startup scripts seem to run  
in this order:

% service -e | grep zfs

The zfsbe service handles mounting of the active boot environment.  The zfs  
service mounts the rest of the ZFS filesystems (the non-"noauto" ones, that  
is).  (The zfsd service runs the ZFS fault management daemon, driven by  
devd events.)

You could run these scripts manually via "/bin/sh -x" to trace what they  
are doing.

I hope this helps.



More information about the freebsd-questions mailing list