A more secure approach of jail establishment. It could be included in jail chapter of fbsd handbook
George Mamalakis
mamalos at eng.auth.gr
Wed Feb 10 12:44:51 UTC 2010
Dear all,
After working with jails for a long time, I ended up with a quite
different approach (architecture) of setting up jails, that seems more
secure than the one proposed in the freebsd handbook.
The concept is very similar to that of handbook's chapter 15.6.1, only
it involves a few more concepts. I'll try to be descriptive and
understandable.
Instead of having the mroot directory (mounted ro) containing the base
system, and the s directory containing the different configs of each
different jail (mounted rw), I introduced one more directory, let's call
it t. Now ls -la /jails/j/mroot gives:
total 32
drwxr-xr-x 14 root wheel - 512 Sep 24 18:16 ./
drwxr-xr-x 10 root wheel - 512 Feb 10 13:17 ../
drwxr-xr-x 2 root wheel - 1024 Sep 24 17:56 bin/
drwxr-xr-x 7 root wheel - 512 Sep 24 17:56 boot/
dr-xr-xr-x 2 root wheel - 512 Sep 24 17:56 dev/
lrwxr-xr-x 1 root wheel - 5 Sep 24 18:15 etc@ -> s/etc
drwxr-xr-x 3 root wheel - 1536 Sep 24 17:56 lib/
drwxr-xr-x 2 root wheel - 512 Sep 24 17:57 libexec/
drwxr-xr-x 2 root wheel - 512 Sep 24 17:56 media/
dr-xr-xr-x 2 root wheel - 512 Sep 24 17:56 proc/
drwxr-xr-x 2 root wheel - 2560 Sep 24 17:56 rescue/
lrwxr-xr-x 1 root wheel - 6 Sep 24 18:15 root@ -> s/root
drwxr-xr-x 2 root wheel - 512 Sep 24 18:14 s/
drwxr-xr-x 2 root wheel - 2560 Sep 24 17:57 sbin/
lrwxr-xr-x 1 root wheel - 11 Sep 24 17:56 sys@ -> usr/src/sys
drwxr-xr-x 2 root wheel - 512 Sep 24 18:14 t/
lrwxr-xr-x 1 root wheel - 5 Sep 24 18:15 tmp@ -> t/tmp
drwxr-xr-x 14 root wheel - 512 Sep 24 18:50 usr/
lrwxr-xr-x 1 root wheel - 5 Sep 24 18:16 var@ -> t/var
And ls -la /jails/j/mroot/usr gives:
drwxr-xr-x 14 root wheel - 512 Sep 24 18:50 ./
drwxr-xr-x 14 root wheel - 512 Sep 24 18:16 ../
lrwxr-xr-x 1 root wheel - 15 Sep 24 18:15 X11R6@ -> ../s/usr/-X11R6
drwxr-xr-x 2 root wheel - 7168 Sep 24 17:57 bin/
drwxr-xr-x 2 root wheel - 512 Sep 24 17:56 games/
drwxr-xr-x 46 root wheel - 5120 Sep 24 17:56 include/
drwxr-xr-x 6 root wheel - 11776 Sep 24 17:57 lib/
drwxr-xr-x 3 root wheel - 12288 Sep 24 17:57 lib32/
drwxr-xr-x 5 root wheel - 512 Sep 24 17:56 libdata/
drwxr-xr-x 5 root wheel - 1536 Sep 24 17:57 libexec/
lrwxr-xr-x 1 root wheel - 14 Sep 24 18:15 local@ -> ../s/usr-local
drwxr-xr-x 2 root wheel - 512 Sep 24 17:56 obj/
drwxr-xr-x 68 root wheel - 1536 Sep 24 18:50 ports/
drwxr-xr-x 2 root wheel - 5120 Sep 24 17:57 sbin/
drwxr-xr-x 25 root wheel - 512 Sep 24 17:56 share/
drwxr-xr-x 22 root wheel - 1024 Sep 24 18:14 src/
As we can see, only tmp and var is linked to t, while all other symlinks
are the same with the configuration proposed in the handbook. The
difference is the following: In my steup, s is mounted ro and t is
mounted rw. In this way, jail's /etc is mounted ro, /usr/ is mounted ro,
which means that /usr/local is mounted ro, which means that /s/portbuild
is mounted read onlty, which means that /root is mounted read only, and
so on... So, all ports installed on the system, and deamons running
through /usr/local are now read only and trojaned versions of binaries
are much more difficult to be installed if the jail is compromised.
In order to be able to manage those jails, I did the following trick:
# cat /etc/fstab
/dev/da0s1a / ufs rw 1 1
/dev/da0s1b none swap sw 0 0
/dev/acd0 /cdrom cd9660 ro,noauto 0 0
# Jails
/jails/j/mroot /jails/j/bind nullfs ro 0 0
/jails/j/mroot /jails/j/heimdal nullfs ro 0 0
/jails/j/mroot /jails/j/ldap nullfs ro 0 0
/jails/s/ldap /jails/j/ldap/s nullfs ro 0 0
/jails/s/bind /jails/j/bind/s nullfs ro 0 0
/jails/s/heimdal /jails/j/heimdal/s nullfs ro 0 0
/jails/t/heimdal /jails/j/heimdal/t nullfs rw 0 0
/jails/t/bind /jails/j/bind/t nullfs rw 0 0
/jails/t/ldap /jails/j/ldap/t nullfs rw 0 0
# Manage jails
/jails/j/mroot /jails/manage/bind nullfs rw 0 0
/jails/j/mroot /jails/manage/heimdal nullfs rw 0 0
/jails/j/mroot /jails/manage/ldap nullfs rw 0 0
/jails/s/ldap /jails/manage/ldap/s nullfs rw 0 0
/jails/s/bind /jails/manage/bind/s nullfs rw 0 0
/jails/s/heimdal /jails/manage/heimdal/s nullfs rw 0 0
/jails/t/heimdal /jails/manage/heimdal/t nullfs rw 0 0
/jails/t/bind /jails/manage/bind/t nullfs rw 0 0
/jails/t/ldap /jails/manage/ldap/t nullfs rw 0 0
devfs /jails/manage/heimdal/dev devfs rw 0 0
devfs /jails/manage/bind/dev devfs rw 0 0
devfs /jails/manage/ldap/dev devfs rw 0 0
In my setup I have three jails: ldap, heimdal and bind. mroot is monted
in each jail's root folder, and the s and t directories of each jail are
null_mounted on each jail's s and t folder respectively. I introduced
another folder, /jails/manage which has the analogous usage to /jails/j
folder, only this folder's subfolders are not used as jails. I chroot to
each folder when I want to have write access on it's corresponding jail,
and I can perform all my administrative operations (if you notice, even
mroot is mounted rw in this chroot, so changes in base disto can be
performed - eg when installing perl where you are asked to change
/usr/bin/perl). This is the reason why I mount devfs on these chroots.
As a result, with this setup, only /var and /tmp are mounted rw in each
jail, and all other filesystems are mounted ro. Management (meaning jobs
that require write access, not starting and/or stopping services) is
achieved by chrooting to each jail's corresponding chroot.
I am working with this setup for more than 4 months without having any
problems; I thought that I could propose it as another paragraph in
chapter 15.6 of freebsd handbook, as an even more secure jail setup,
instead of just publishing it to some blog,forum,etc.
Thank you all for your time.
--
George Mamalakis
IT Officer
Electrical and Computer Engineer (Aristotle Un. of Thessaloniki),
MSc (Imperial College of London)
Department of Electrical and Computer Engineering
Faculty of Engineering
Aristotle University of Thessaloniki
phone number : +30 (2310) 994379
More information about the freebsd-doc
mailing list