to jail or not to jail

[Dave Cottlehuber]

On Sun, 2 Jun 2019, at 10:00, Matthew Seaman wrote:
>
> For letsencrypt purposes, I use a DNS-01 challenge because that seemed
> to make the most sense given I wasn't going to deploy most certs on web
> servers.  Then I just wrote a custom deploy hook script to copy certs
> into the jail filesystems and restart servers.  Although I've created at
> lease a separate ZFS for each jail, I haven't gone down the route of
> using 'zfs jail ...' to hide them from the main host system, as it makes
> copying things into jails from the host that much easier.

Minor clarification - when a jailed zfs dataset is mounted inside a running
jail, it is accessible from the host server.

This host server has a zroot/jailed parent to ensure that jailed datasets
can't inherit a mountpoint from the host system, and also to remind me
that they are indeed supposed to be jailed and not locally available:

# zfs list -o canmount,mounted,readonly,name,jailed -r zroot/jailed
CANMOUNT  MOUNTED  RDONLY  NAME                         JAILED
     off       no     off  zroot/jailed                    off
      on      yes     off  zroot/jailed/couchdb2            on
      on      yes     off  zroot/jailed/couchdb2/views      on
      on      yes     off  zroot/jailed/mu                  on
      on      yes     off  zroot/jailed/www                 on

# ls /jails/www/var/www/
...

It's only when the jail is not running, that the dataset is not available
to the host system:

# zfs mount zroot/jailed/www
cannot mount 'zroot/jailed/www': dataset is exported to a local zone

But you can deliberately bypass this temporarily via:

# mount -t zfs zroot/jailed/www /mnt

I wrote a minimal example of using "raw" jails as opposed to iocage
driven jails a few years ago, this may be of use as it shows how
to provide DNS, pf.conf settings, etc behind a single NAT IP:

https://git.sr.ht/~dch/diy-jails/tree/master/zjail only try it on a test VM!

If applications support it, you can run a jail that only contains a single
process - there's no inherent need for cron, syslog (use the host's  syslog
directly via UNIX socket or via UDP), sshd, ntpd, sendmail etc.

> think about using vimage jails on 12.0, as that makes the jails seem a
> lot more like just regular VMs, and gives you the ability to effectively
> create a private virtual switch inside your server, rather than having
> services appear on external interfaces.  Beware though that there are
> currently some quite severe bandwidth limitations on this sort of
> internally virtualized networking under FreeBSD, so this is not suitable
> for a high-traffic system.

Matthew, anything you can point me to about this limitation?

A+
Dave