ZFS and Jail :: nullfs mount :: nothing visible from host :: solved [partial]

[SK]

On 16/12/2016 13:15, Alexander Leidinger wrote:
> Quoting SK <fbstable at cps-intl.org> (from Mon, 12 Dec 2016 17:13:27 
> +0000):
>
>> b) Alexander, I am still not able to do snapshot or any other action 
>> from within my jail. My understanding is that you are using ezjail, 
>> which might be doing something that my regular jail creation is 
>> ommitting. If you do not mind sharing your configuration steps, I can 
>> try to reproduce it at this end. If it is exactly as it is on the 
>> site you pointed to earlier, please let me know, I will follow that 
>> verbatim (even though I do not remember seeing anything there that I 
>> have not tried already, but I might be mistaken).
>
> Do you use quotas on the datasets you want to add to the jail? If yes, 
> try without. The man-page of zfs tells that this value can not be 
> changed (but from the wording I would expect hat an already set value 
> should work).
>
Hi Alexander, short answer, NO, I have not used quota yet

> ezjail is just a shell script which simplifies some things with jails. 
> For a particular jail where I can manage the datasets which are handed 
> over to the jail I have those settings in ezjail which correspond to 
> the settings you can specify in jail.conf:
> ---snip---
> export jail_xyz_leidinger_net_devfs_ruleset="17"
> export jail_xyz_leidinger_net_zfs_datasets="space/something"
> export jail_xyz_leidinger_net_parameters="allow.mount allow.mount.zfs 
> enforce_statfs=1"
> ---snip---
> Check if you have allow.mount and allow.mount.zfs for the jails in 
> question.
>
Yes, those are in place.

> Note, "space/something" is not the root of the jail, it's a seperate 
> dataset. Do not add the root of the jail as a dataset. Example bellow.
>
I think that might have been the cause for it to not work for me. I had 
space/something/jailRoot, and defined that as the dataset for the jail

> devfs.rules part:
> ---snip---
> [devfsrules_unhide_zfs=12]
> add path zfs unhide
>
> [devfsrules_jail_withzfs=17]
> add include $devfsrules_hide_all
> add include $devfsrules_unhide_basic
> add include $devfsrules_unhide_login
> add include $devfsrules_unhide_zfs
> ---snip---
>
> The rc.conf inside this jail:
> ---snip---
> zfs_enable="YES"
> ---snip---
>
These are all in place as expected.

> For one of the filesystems I have set "zfs allow" permissions, but 
> just that a specific user in the jail can do something on those FS 
> without the need to switch to root. So as long as you try to do a zfs 
> create/snapshot with an user with UID 0 inside the jail, the "zfs 
> allow" part doesn't come into play.
>
> So assume space/jails/xyz.leidinger.net/ to be the dataset which 
> contains the root of the jail but is not attached/attributed to the 
> jail itself. space/jails/xyz.leidinger.net/data with mountpoint=none 
> to be attributed ("zfs jail xyz space/jails/xyz.leidinger.net/data") 
> to the jail (similar to the "space/something" in the ezjail config 
> above, I have some iocage-managed jails were this works). In this case 
> you should be able to do from inside the jail "zfs create -o 
> mpuntpoint=/mnt space/jails/xyz.leidinger.net/data/test".
>
hmmm, I'm slightly confused at this point. Let me see if I can clarify 
that in my brain

If I understand you correctly, what you are suggesting is, the dataset 
used by the jail itself for its root/base cannot be "worked on" from 
within the jail, but if I define a different dataset (under the same 
branch below the jail dataset), and attribute it to the jail, then I can 
manipulate that "other" dataset. Could you please confirm if I 
understood it correctly?

>> And now to everyone, I am still confused about zfs set jailed=on. As 
>> I mentioned on my previous emails, as soon as I do that, the dataset 
>> vanishes from the host system (as I understand, that is expected 
>> behaviour). Then the jail fails as it is unable to mount /dev, /proc
>
> From the zfs man page:
> ---snip---
>      After a dataset is attached to a jail and the jailed property is 
> set, a
>      jailed file system cannot be mounted outside the jail, since the 
> jail
>      administrator might have set the mount point to an unacceptable 
> value.
> ---snip---
>
> So yes, it is expected that it "vanishes", but it should be visible 
> from the parent host at the place inside the jail FS subtree were it 
> is mounted there (after setting the mountpoint of the dataset).
>
I think what you are trying to tell here is, unless and until that 
"vanished" dataset is put to use (mounted) from inside the jail, it will 
remain vanished/unusable from the host itself; however, once that 
dataset is put to use, the host system should be able to "see" and maybe 
even work on that dataset. Could you please confirm if I understood you 
correctly?

>> and so on. I have to change jail.conf and comment out mount.devfs and 
>> mount.procfs -- but than in turn makes /dev/zfs unavaulable and I 
>> cannot do anything from inside the jail.
>
> Could it be that you try to attribute the root of the jail as a 
> dataset into the jail?
>
Yes, I did. But now that I have a few more slightly different 
explanation from you, once you confirm if my understanding is correct, I 
am willing to give this a try.

Thanks again for your suggestions and guidlines.

Have a lovely weekend.
SK