misc/186345: FreeBSD 10 Installer creates weird ZFS volume layout for /usr

[John-Paul Bader]

>Number:         186345
>Category:       misc
>Synopsis:       FreeBSD 10 Installer creates weird ZFS volume layout for /usr
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          change-request
>Submitter-Id:   current-users
>Arrival-Date:   Sat Feb 01 15:40:00 UTC 2014
>Closed-Date:
>Last-Modified:
>Originator:     John-Paul Bader
>Release:        10.0 Release
>Organization:
>Environment:
FreeBSD awesome 10.0-RELEASE FreeBSD 10.0-RELEASE #0 r260789: Thu Jan 16 22:34:59 UTC 2014     root at snap.freebsd.org:/usr/obj/usr/src/sys/GENERIC  amd64

>Description:
After installing a fresh FreeBSD 10.0 Release with ZFS root on my server I created a snapshot with the -r option. Then I created a file in /usr/local and wanted to rollback to the previous state but noticed that nothing happened. 

After a bit of investigation I found out that zfs list claims that /usr is a separate volume which gets its own snapshot entries but in reality it is not mounted and instead located in zroot/ROOT/default 

So in order to rollback /usr/local… I'd have to rollback zroot/Root/default. 

 zfs get all zroot/usr reports that /usr is not mounted and the canmount option is set to off.

This creates a lot of confusion and is counter intuitive. If you carefully compare the output of zfs list and mount you can see the problem maybe a bit better:

$> mount
zroot/ROOT/default on / (zfs, local, noatime, nfsv4acls)
devfs on /dev (devfs, local, multilabel)
zroot/tmp on /tmp (zfs, local, noatime, nosuid, nfsv4acls)
zroot/usr/home on /usr/home (zfs, local, noatime, nfsv4acls)
zroot/usr/ports on /usr/ports (zfs, local, noatime, nosuid, nfsv4acls)
zroot/usr/src on /usr/src (zfs, local, noatime, noexec, nosuid, nfsv4acls)
zroot/var on /var (zfs, local, noatime, nfsv4acls)
zroot/var/crash on /var/crash (zfs, local, noatime, noexec, nosuid, nfsv4acls)
zroot/var/log on /var/log (zfs, local, noatime, noexec, nosuid, nfsv4acls)
zroot/var/mail on /var/mail (zfs, local, nfsv4acls)
zroot/var/tmp on /var/tmp (zfs, local, noatime, nosuid, nfsv4acls)

Mount shows no /usr volume but when you look at zfs list you get:

$> zfs list
NAME                 USED  AVAIL  REFER  MOUNTPOINT
zroot               2.17G   103G   144K  none
zroot/ROOT           857M   103G   144K  none
zroot/ROOT/default   857M   103G   857M  /
zroot/tmp            192K   103G   192K  /tmp
zroot/usr           1.33G   103G   144K  /usr
zroot/usr/home       184K   103G   184K  /usr/home
zroot/usr/ports      813M   103G   813M  /usr/ports
zroot/usr/src        545M   103G   545M  /usr/src
zroot/var           1.27M   103G   644K  /var
zroot/var/crash      148K   103G   148K  /var/crash
zroot/var/log        212K   103G   212K  /var/log
zroot/var/mail       144K   103G   144K  /var/mail
zroot/var/tmp        152K   103G   152K  /var/tmp

when listing the snapshots again you will get an entry for /usr

zfs list -t snapshot
NAME                      USED  AVAIL  REFER  MOUNTPOINT
zroot at eins                   0      -   144K  -
zroot/ROOT at eins              0      -   144K  -
zroot/ROOT/default at eins     8K      -   857M  -
zroot/tmp at eins               0      -   192K  -
zroot/usr at eins               0      -   144K  -
zroot/usr/home at eins          0      -   184K  -
zroot/usr/ports at eins         0      -   813M  -
zroot/usr/src at eins           0      -   545M  -
zroot/var at eins             96K      -   644K  -
zroot/var/crash at eins         0      -   148K  -
zroot/var/log at eins         84K      -   212K  -
zroot/var/mail at eins          0      -   144K  -
zroot/var/tmp at eins           0      -   152K  -

but in reality /usr and all its sub directories which are not a dedicated volume are under zroot/ROOT/default.

Can you please clean up this mess if there are no real reasons to keep it that way?
>How-To-Repeat:
Install a fresh FreeBSD 10 with ZFS root filesystem with bsdinstall. Make a snapshot, create a file in /usr and try to rollback to the snapshot with zfs rollback zroot/usr at snapshot - /usr will remain the same. Instead do zfs rollback zroot/ROOT/default and the file in /usr will disappear.
>Fix:


>Release-Note:
>Audit-Trail:
>Unformatted: