Neutered devices in jails (per FS flag?)

[Mateusz Guzik]

On Wed, Sep 16, 2015 at 11:30:33AM -0500, Kevin Day wrote:
> We’re currently using jails to allow servers to copy backups of themselves to a central backup server. The problem we’re having is with mknod/devices. Currently jails don’t allow device files to be created, which makes sense - you don’t want them to be able to bypass the jail by opening /dev/kmem or something. We want jails to be able to create device files, just not be able to open/use them.
> 
> Has anyone given any thought to changing this behavior? Allowing jails to create/manipulate device files, but not actually opening them? I.e. instead of returning EPERM on creating the device, instead return EPERM on opening it? This would likely need to be a filesystem flag, because jails still require some devices to work (a separate devfs mount or something). We could make the jail’s /dev read only or use devfs so those devices still work, but have the parent jail directory with a “noopendev” flag or something similar.
> 
> Has anyone gone down this path before?
> 

Let's lay down some facts to make things clear.
1. device nodes on regular filesystems are not treated as devices by the
  kernel
2. device visibility in devfs is controlled with appropriate rules.
mknoding a device will make it appear, regardless of presence of a
'hide' rule

So, mknod in question /on devfs/ would be useful to make stuff reappear
if it was deleted by accident, i.e. its a nice little feature. Allowing
jailed root to make explicitly hidden devices visible is a complete
non-starter (regardless of whether it is allowed to use them) so this
would have to be plugged.

Allowing jailed root to mknod on regular filesystems by defualt is also
a non-starter because said filesystems may be exported with nfs and the
other party possibly forgot about nodev and actually respects device
nodes.

Further, you can mknod more than just a device so that would have to be
audited.

So, to summarize, this can be done.

So what is needed for such a feature to hit the tree:
1. it would have to be an opt-in thingy (similar to how e.g. sysvipc is
handled) - a trivial change
2. arbitrary device creation on devfs would have to be disabled if the
user is jailed - likely a trivial change
3. someone has to audit mknod - unclear

I'm not up to the task at the moment though.

I'm happy to take a look at patches for 1 and 2, but I'm not committing
anything without point 3 being executed (and I'm not touching it for
now).

-- 
Mateusz Guzik <mjguzik gmail.com>