ZFS and jailed environments -- best practice?

[Mahlon E. Smith]

I've been playing with mixing up ZFS and jailed environments under
8.0RC, and I've hit a point where I'm just kind of wondering how
everyone else is doing it.  I wanted to do this to take advantage of
delegated administration -- I want users inside a jail to be able to
control snapshot/rollback in their own homedir.

I'll break this up into what I did to get it working (I can't seem to
find a good step by step out there yet), and where I think I'm running
into what could be potential trouble.

First off, sysctl variables.

    security.jail.enforce_statfs=0
    security.jail.mount_allowed=1
    vfs.usermount=1

I've always run jails with with enforce_statfs=1 or enforce_statfs=2.  I
honestly don't see why that wouldn't work for ZFS stuff too, but in the
interests of following instructions (the zfs man page), I set it to 0.

Next, the 'zfs' dev node needs to be accessible from inside the jail.
So I created an /etc/devfs.rules file with the following:

    host# cat /etc/devfs.rules 
    [zfsenable=10]
    add path 'zfs' unhide 

...and added the ruleset to the jail config in rc.conf:

    jail_zfstest_devfs_ruleset="zfsenable"

So far so good, the jail gets a /dev/zfs, and I can issue zfs commands.
I get 'no datasets available' from within the jail, which is exactly
what I'd expect.

So, tank/jails/jail1 is a ZFS volume, and I want tank/jails/jail1/home
to be under the control of the jail, and mounted at /home inside of it.


I stop the jail and unmount the home volume.

    host# zfs umount tank/jails/jail1/home

Then enabled 'jailed mode' on the volume, and start the jail back up.

    host# zfs set jailed=on tank/jails/jail1/home

In the host, lets just say the JID is 8.

    host# /sbin/zfs jail 8 tank/jails/jail1/home


From that point, it appears that the host thinks that volume is not
under its own control.  (good!)

    host# zfs mount tank/jails/jail1/home
    cannot mount 'tank/jails/jail1/home': dataset is exported to a local zone

Whew, okay.  Back into the jail.

    jail# zfs set mountpoint=/home tank/jails/jail1/home
    jail# zfs mount -a
    jail# zfs allow -s @homedir create,clone,mount,rollback,snapshot,send,receive,compression,checksum,quota,readonly,destroy tank/jails/jail1/home
    jail# zfs allow -u user1 @homedir tank/jails/jail1/home/user1

... and by god, it works.  Yay!


Here are the weird parts, or parts that make me feel like I'm not doing
something correctly.

1) From the host now -- I've got two /home partitions mounted when
displaying a 'df'.  They -appear- to do the right thing... /home on the
host is correct when getting a listing, and /home in the jail is also
correct.  But I can't help but feel like this is asking for trouble, or
will eat the delicious data at some point.

2) What the heck is the procedure for automating this on boot?  Roll
your own?  The JID shuffles, of course.  I could easily whip up some
zfs jail `jls | awk '/jail1/ { print $2 }' ... junk, but where would
I put something like that? jail_afterstart0="" seems to load things
in the context of the jail, not the host.  And then I'd have to set
canmount=noauto on that home volume, and mount it manually from within
the jail via some startup script?  Seems... like a pain in the ass for
what is otherwise a pretty blissful setup.

Really, I'm not sure what's right, what's stable, and what won't make me
totally regret doing this later.  :)

Advice, discussion, or pointers elsewhere are all appreciated!

-Mahlon

--
Mahlon E. Smith  
http://www.martini.nu/contact.html

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 155 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-questions/attachments/20091126/12701d5a/attachment.pgp