Avoiding source code on production servers

[Paul Schenkeveld]

On Fri, May 22, 2009 at 11:06:49AM +0200, Tonix (Antonio Nati) wrote:
> I'm in the phase of planning my new generation of FreeBSD servers, and I 
> would love to make them more easy to upgrade.
> Main problem I have currently is I do not want any source code on 
> production server, so freebsd-update is welcome, but... what about 
> packages?
> I would use packages, but they are not easy to upgrade, while ports can be 
> easy to upgrade, but need to have sources an servers.
> 
> What do you suggest me? What is currently done on other environments?

I've spent a lot of time over the last three years trying to automate
maintenance of source-free servers.  Ports are a real challenge.

Other work with embedded systems (Soekris) has brought me the idea of
using nanobsd(8) for servers.  Although it may sound strange at first,
experiments I'm currently undertaking give very promising results.

The operating system and all ports are put into a read-only mounted
root filesystem.  /etc is a malloc-backed memory filesystem which gets
filled by "standard" /etc contents part copied into /conf/base/etc in
the root filesystem and then gets overlayed by modified files which
are saved in a separate /cfg filesystem that you can mount read-write
when changing configuration.

/var, /home and other filesystems with user data are normal rw
filesystems.

Each server has two slices holding a root filesystem each, one is active
and the other will be used to upload a new image when upgrading or
adding software.  After upgrading this alternate root slice you have
to reboot the server so if you cannot tolerate a reboot, nanobsd is not
for you.  A roll-back is very easy if the new root does not satisfy you,
just reboot and select the old slice to boot.

Many of my servers have all applications hidden in jails, this makes
this solution even easier as the host operating system ususally is
very small on such servers.  Each jail can be maintained and upgraded
in a similar way, I keep a spare jail around to prepare the upgeade of
/ and /usr parts of application jails, stopping and restarting an
application jail with the new /+/usr slice upgrades the software in
the jail, rollbacks are easy as well.

-- Paul Schenkeveld