kern/117829: [unionfs] can't mount devfs on top of unionfs anymore

[Sebastian Klemke]

>Number:         117829
>Category:       kern
>Synopsis:       [unionfs] can't mount devfs on top of unionfs anymore
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Mon Nov 05 00:50:02 UTC 2007
>Closed-Date:
>Last-Modified:
>Originator:     Sebastian Klemke
>Release:        6-STABLE
>Organization:
>Environment:
FreeBSD roterstern.xberg.nerdheim.de 6.3-PRERELEASE FreeBSD 6.3-PRERELEASE #0: Sun Nov  4 03:22:17 CET 2007     root at roterstern.xberg.nerdheim.de:/usr/obj/usr/src/sys/ROTERSTERN  i386
>Description:
My jail setup is broken due to unionfs trouble: I'm using unionfs to overlay jail-specific stuff over a world. The trouble is that devfs on top of unionfs does not seem to work anymore in 6-STABLE (I suspect the Oct 23, 2007 changes in sys/fs/unionfs, but haven't had the time to test with an older checkout). This is what I got in my rc.conf:

jail_enable="YES"
jail_devfs_enable="YES"
jail_mount_enable="YES"
jail_interface="fxp0"
jail_devfs_ruleset="devfsrules_jail"
jail_set_hostname_allow="NO"
jail_socket_unixiproute_only="YES"
jail_sysvipc_allow="YES"
jail_list="svn"
jail_svn_rootdir="/home/jails/svn"
jail_svn_hostname="svn.xberg.nerdheim.de"
jail_svn_ip="192.168.0.75"

When starting the jail via /etc/rc.d/jail start, the unionfs gets mounted and then on top of the unionfs, devfs is mounted. I can confirm with mount and sh -x that devfs is really mounted on top of unionfs, no errors. But then, when rulesets are applied to the new devfs I get:

+ debug devfs_set_ruleset: setting ruleset (4) on mount-point (/home/jails/svn/dev)
+ /sbin/devfs -m /home/jails/svn/dev ruleset 4
devfs ruleset: ioctl DEVFSIO_SUSE: Inappropriate ioctl for device
+ warn devfs_set_ruleset: unable to set ruleset 4 to /home/jails/svn/dev
+ [ -x /usr/bin/logger ]
+ logger /etc/rc.d/jail: WARNING: devfs_set_ruleset: unable to set ruleset 4 to /home/jails/svn/dev
+ echo /etc/rc.d/jail: WARNING: devfs_set_ruleset: unable to set ruleset 4 to /home/jails/svn/dev
/etc/rc.d/jail: WARNING: devfs_set_ruleset: unable to set ruleset 4 to /home/jails/svn/dev
+ return 1
+ devfs -m /home/jails/svn/dev rule applyset
devfs rule: ioctl DEVFSIO_SAPPLY: Inappropriate ioctl for device

When I examine the mountpoints of the devfs with ls, it looks like a normal directory without devfs mounted. I'm using -o below for the unionfs and everything written into /home/jails/svn/dev ends up getting written to the real, physical /home/jails/svn/dev that resides on my /home filesystem, like it would be expected if devfs wasn't mounted. But actually devfs is mounted, I can even umount it.
>How-To-Repeat:
1. mkdir /tmp/a /tmp/b /tmp/b/dev
2. mount -t unionfs -o below /tmp/a /tmp/b
3. mount -t devfs dev /tmp/b/dev
4. now examine b/dev: It's possible to umount /tmp/b/dev, but nothing is really there
>Fix:


>Release-Note:
>Audit-Trail:
>Unformatted: