FreeBSD Jails section for Handbook
Khairil Yusof
kaeru at pd.jaring.my
Wed Jun 9 05:36:39 UTC 2004
This is a first go, and I would like to have comments on what such a
section needs included/removed before it can be good enough for
inclusion in the Handbook or as an article.
http://www.inigo-tech.com/Members/kaeru/articles/freebsd/freebsd-jails
I've attached a plain text file in structured text format for now.
-------------- next part --------------
DRAFT
FreeBSD Jails
FreeBSD jails are self contained environments, which share resources on the
host system but have their own ip address, configuration and programs.
Essentially a jail can be used to run a complete virtual FreeBSD system
(also known as virtual servers) without high resource requirements of
complete virtual environments such as VMWare.
Since they are self contained environments, jails are useful when
there is a need for different server configurations that do not conflict
with each other or, not to mess up a system when testing development
versions of ports or programs. Currently the focus of this article are
jails for a development environment, and will not concentrate too much on
security. The setup will be that of a full FreeBSD distribution, which has
everything the default base system has.
Examples of jails as virtual servers usage
* Development versions of ports which can conflict with
stable versions such as Gnome development ports.
* Different conflicting versions of servers or programs.
* Ports testing, source installs
Where mistakes or lack of a packaging can result in a messy state.
It is easy to wipe out a jail directory without affecting the host
system.
* Shared hosting or sharing a physical server within a company.
**This article does not deal with security just yet to
setup secure virtual servers**
Creating a jail
Building and Installing
The jail(8) man page has good clear documentation on this, and this the
instruction from the man page on setting up a directory tree containing an
entire FreeBSD distribution.
For this example the host environment will have an ip address of
192.168.1.1 Jail directory names are usually set as IP addresses while
another naming convention that can be used is hostnames.
The jail will be setup with an ip address of 192.168.1.2 and in the
directory /usr/jail/192.168.1.2
The following commands should be run as the root user or by a regular
user with the proper sudo(8) permissions.
For csh/tcsh:
::
setenv D /here/is/the/jail
For sh/bash/ksh users:
::
export D=/here/is/the/jail
On 5.X:
::
cd /usr/src
mkdir -p $D
make world DESTDIR=$D
cd etc
make distribution DESTDIR=$D
mount_devfs devfs $D/dev
cd $D
ln -sf dev/null kernel
On 4.X:
::
cd /usr/src
mkdir -p $D
make world DESTDIR=$D
cd etc
make distribution DESTDIR=$D
mount_devfs devfs $D/dev
cd $D
ln -sf dev/null kernel
cd $D/dev
sh MAKEDEV jail
mkdir $D/stand
cp /stand/sysinstall $D/stand
Notes
If you have already built world, you can replace **make world** with
**make installworld** on the fourth line to avoid recompiling everything
again.
**WARNING**
With **make distribution** be **very careful** that
DESTDIR environment variable is set. If it is not, it defaults to overwriting
your host system's base setup (rc.conf, passwd etc.). If in doubt,
type it out manually.
Setting up the host environment
Setup daemons to listen only on host ip address
On the host, any daemons that listen on all ports, need to be removed
if they will affect those in the jails. This is because, jail daemons
all bind onto the same network interface through use of network aliases
on the host. We need to either disable some services, or configure
then to listen only on the host ip address.
::
inetd_flags="-wW -a 192.168.1.1"
sendmail_enable="NO"
On 5.x
::
rpcbind_enable="NO"
On 4.x
portmap_enable="NO"
Where 192.168.1.1 is the host ip address, all services run out of inetd
will now automatically only listen on that address. We have also set
the sendmail_enable flag to ?NO? to prevent sendmail from listening on
external host interfaces. This will still allow a sendmail process to
start and bind to the localhost address of the host environment so as
not to interupt programs and scripts which mail their output just as
the standard nightly cron jobs. Lastly, the RPC port mapper has been
disabled. The jail host cannot be an NFS server or client because it
cannot be configured to listen to a specific IP address.
We will then need to list out any other daemons that are listening
on all ip addresses.
::
sockstat -l4
Any services that have *: and not a specific ip will be listening
on all available ip addresses. This is not good, as it may conflict
with one of the services listening in our jails.
sshd
In /etc/ssh/sshd_config, add or change the ListenAddress as
below substituting the ip address with that of your host ip.
::
ListenAddress 192.168.1.1
The following services are optional services that may be running on
your machine though they are not enabled by default. Here is how to
have them listen to a single IP address by default.
bind
/etc/nameddb/named.conf
Add the listen-on option to only listen on specific ip addresses.
::
listen-on { 192.168.1.1;
}
Starting and setting up the jail environment
Starting the jail for the first time.
Now that the host environment has been setup it must be told to accept
network traffic addressed to our jail's IP address. Once that is
accomplished we can startup the jail for the first time.
::
ifconfig ed0 inet alias 192.168.1.2/3
jail /usr/jail/192.168.1.2 gnome 192.168.1.2 /bin/csh
In the ifconfig(8) command, replace the "ed0" with the name of your
network interface on the host system. This will allow the host
environment to accept packets that are addressed to your jail's
address.
The jail command will start your jail, give it the hostname "gnome",
the IP address 192.168.1.2, and drop you to a csh prompt. You will now
be interacting directly with the jail host.
Configuring the jailed system.
The next few steps are the required post-install configuration that is usually
done at the end of a standard installation.
* We need to create an empty /etc/fstab
touch /etc/fstab
* Run newaliases(1) to quell sendmail(8) warnings.
* Configure /etc/resolve.conf so that name resolution will work
properly. For this example, set it to the host IP address where we
have a dns server running.
::
nameserver 192.168.1.1
* In rc.conf put these variables in. Enabling SSH will make it easier
for us to access and run programs on the virtual server.
::
sshd_enable="YES"
network_interfaces=""
The following options may also be added if these services are not
relevant to the operation of the jail.
::
sendmail_enable="NO"
inetd_flags="NO"
On 5.x:
::
rpcbind_enable="NO"
On 4.x:
::
portmap_enable="NO"
We turned off sendmail from receiving mail from all but the
localhost, in this case, the jail's own localhost. If you are not
going to be providing network services from your jail you might as
well turn off inetd. SSH will allow you to use SFTP for file
transfers since FTP will now be shutoff. The RPC port mapper has
been disabled but a jail itself MAY still use or serve NFS services
if actually needed unlike the host environment.
The last steps can be done from either the command line or through sysinstall(8).
* Set password for root with passwd(8)
* Add accounts for users with adduser(8)
* Configure your timezone
Installing basic ports and packages
At this point you can begin to add programs through packages or via
the ports system that are needed for the functionality of your jail.
You cannot use sysinstall(8) from within a jail to add packages or
install the ports tree from remote locations.
To install from packages:
For csh/tcsh:
::
setenv PACKAGEROOT ftp5.us.FreeBSD.org
For sh/bash/ksh users:
::
export PACKAGEROOT=ftp5.us.FreeBSD.org
::
pkg_add -r portupgrade
pkg_add -r pdksh
etc...
Change The PACKAGEROOT to our desired FTP mirror site. By setting that and using pkg_add(8) with the "-r" option pkg_add will download the latest package for your release.
To install from the latest ports:
::
fetch ftp://ftp5.us.freebsd.org/pub/FreeBSD/ports/ports/ports.tar.gz
cat ports.tgz | tar --unlink -xpzf - -C /usr
**WARNING**
If you have installed packages and then decide to start installing
from the latest ports, run portupgrade -uarR directly after you
have created the ports tree so that your currently installed
package versions will be in sync with your ports tree.
Managing Jails
Using system rc scripts to start and stop jails
In 5.X /etc/defaults/rc.conf has a well documented section on
configuring jails. We can control our jails from the host system using
this script after we have added the appropriate lines in /etc/rc.conf
on the host system.
rc.conf
For our example we will add the following lines to /etc/rc.conf
(on host)
::
jail_enable="yes" # Set to NO to disable starting of any jails
jail_list="gnome" # Space separated list of names of jails
jail_gnome_rootdir="/usr/jail/192.168.1.2" # Jail's root directory
jail_gnome_hostname="gnome.domain.com" # Jail's hostname
jail_gnome_ip="192.168.1.2" # Jail's IP number
jail_gnome_exec="/bin/sh /etc/rc" # command to execute in jail
jail_gnome_devfs_enable="YES" # mount devfs in the jail
jail_gnome_fdescfs_enable="YES" # mount fdescfs in the jail
jail_gnome_procfs_enable="YES" # mount procfs in jail
#jail_gnome_devfs_ruleset="ruleset_name" # devfs ruleset to apply to jail
You should also do the ip aliasing automatically by adding
this line in your rc.conf
ifconfig_fxp0_alias0="inet 192.168.1.2 netmask 0xffffffff"
Starting the jail
Now to start the jail, you only need to issue the command.
If more than one jail is defined in the startup list, all
will be started.
::
/etc/rc.d/jail start
To start a specific jail
::
/etc/rc.d/jail start gnome
Stopping the jail
To stop all jails
::
/etc/rc.d/jail stop
To stop a specific jail
::
/etc/rc.d/jail stop gnome
Checking that your jail is running
To get a list of running jails use the jls(8) command
::
jls
Reusing and Sharing directories
Reusing jails
The steps above to build and setup a jail can of course take a while.
This is not prefferable if you are looking to do a limited test of a
few specific scenarios and then discard the jail once you are done.
You may find it useful to build a basic jail having gone through the
steps above, make any additional configuration edits, and install any
basic ports that you require. From that point you can repeatedly copy
that jail in order to create new ones. To do this, cp -R will not
work. Instead, do the following while the original jail is not
running.
::
mkdir /usr/jail/new
cd /usr/jail/old
tar -cpf - . | tar -C /usr/jail/new -xpf -
Sharing a filesystem between jails
Often you want to share directories over different jails, such as
distfiles for ports to save disk space. One way to do it is via nullfs.
mount_nullfs(8)
::
mount_nullfs /shared/distfiles /usr/jail/192.168.1.2/usr/ports/distfiles
You can automatically mount it at boot time by adding an entry in
/etc/fstab on the host system.
::
/shared/distfiles /usr/jail/192.168.1.2/usr/ports/distfiles nullfs rw 0 0
Network Access
FIXME
Gnome/X11 Desktop specific
Forwarding X through ssh
There are some issues with forwarding X applications via ssh.
Change the entry in /etc/ssh/sshd_config (jail) and set
::
PermitRootLogin yes
X11UseLocalhost no
Notes
I still had some problems with certain applications like gvim.
Also there are X authentification errors if I su and try
to run an X app. If you know of a solution, please let me know.
These problems may only be occuring due to having host X display,
on the same box.
Gnome Desktop and GDM via secondary X Server and XDMCP
GDM and XDMCP
* ssh in as root to your virtual server and run
gdmsetup. Enable XDMCP.
"XDMCP":img:gdmsetup.png
* Run 'gdm' to start up the gdm service.
* switch to a free console and you can now login to your
jail desktop on a secondary X Server.
::
X :1 -query gnome
* To automatically start up gdm in your jail, copy
/usr/X11R6/etc/rc.d/gdm.sh.sample
to
/usr/X11R6/etc/rc.d/gdm.sh
Gnome Development testing
For those interested in helping out with FreeBSD Gnome testing,
you should visit the "FreeBSD Gnome Development FAQ":http://www.FreeBSD.org/gnome/docs/develfaq.html.
There are additional jail and build scripts, available from the
development cvs at marcuscom.com
Authors
* "Khairil Yusof":mailto:kaeru at pd.jaring.my
* 4.x specific content and further editing by tmclaugh at sdf.lonestar.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 187 bytes
Desc: This is a digitally signed message part
URL: <http://lists.freebsd.org/pipermail/freebsd-doc/attachments/20040609/69740396/attachment.sig>
More information about the freebsd-doc
mailing list