ZFS, Jails, network, routing, domains and IP addresses

[Matthew Seaman]

On 09/09/14 14:22, Niklaas Baudet von Gersdorff wrote:
> Hi,
> 
> I am not an educated computer scientist but got in touch with UNIX and
> Linux quite early. Since then I ran several servers and am somehow
> finding my way through the IT world by readings lots of blogs, articles
> and mailing lists about the topic as a hobby. At the moment I am running
> a root server at some provider who I don't like anymore (this has its
> reasons) and would like to switch the provider. Because this will be
> some work in any event I thought about simultaneously switching from
> Debian to FreeBSD since this is something I would like to do for quite
> some time.
> 
> The main reasons for switching to FreeBSD are jails and the ports
> system. My question concerns jails and the set-up I thought about. If
> you have any thoughts about it please just give me some short hints and
> I'll be very happy about that. :-)
> 
> So, the future server has 48 GB of RAM and 2 2TB HDDs. I thought about
> installing FreeBSD 10 with ZFS (on /) mirroring both HDDs. (I already
> did that set-up smaller and virtualized on my desktop machine and that
> worked great.) I would like to use jails since I've got several domains
> to administer and each domain belongs to another friend of mine. Hence,
> they should not get any access to the jail host or other jail clients.
> So, I would like to use jails to virtualize several servers. On every
> host there'll be a Postfix and Apache installation. So, everything stays
> quite simple. Nothing complex.

That should do fine.

If you're going to be building a lot of jails with much the same
software load-out, then you can make it all quite space efficient by
building a template jail and then using ZFS cloning.  Either as a DiY
setup or try ezjail.

Note: don't turn on ZFS deduplication.  It sounds attractive, but you
will need a lot more RAM than you have in order for it to be effective,
and it does entail trading off performance for storage efficiency.

> 1. ZFS and Jails
> 
> It would be cool if I could simplify the process of updating the
> software that is running in every jail. I searched in the web for some
> information and also had a look at the FreeBSD mailing lists. It looks
> like it's quite a popular set-up to create a "base" FreeBSD Jail that is
> cloned with the help of ZFS if there is a new jail needed. The ports
> tree is mounted with a nullfs in every jail so updating the "main" ports
> tree would lead to the software in every jail getting updated. Or am I
> understanding something totally wrongly here?

Read about the -c and -j flags in pkg(8).  Also, I recommend managing
your jails entirely through binary packages, rather than mounting ports
trees everywhere.  You can either use the standard FreeBSD pkg repos, or
build your own with poudriere or indeed a combination of the two.

> While reading I also got the impression there are different methods for
> maintaining Jails with ZFS. I would be very thankful if anyone will
> point out the different approaches that exist (some articles on the net
> seem outdated). Maybe a quick reference to necessary man pages are
> already enough, then I can do further research on my own. :-)
> 
> 2. Jails and routing
> 
> The main question is: Is it possible for the jails' host to distinguish
> between incoming connections depending on the domain look-up they did?
> If it is possible I would like to use as less IP addresses as possible.
> Could be that it's technically not possible at all but I thought there
> is maybe some way to do it and someone knows. The idea is the jails'
> host does something like this: Connection to Domain#1 established so
> everything goes to Jail#1, Connection to Domain#2 established so
> everything foes to Jail#2, ... but the jails and the jails' host use the
> same IP X.

This might be possible, but it's not something that is usually done.
Given you've said the applications you'll be supplying are postfix and
apache, then you should be able to have a small instance of either of
those acting as a reverse proxy in front of your jailed environments
(which can just use some private address space).  You can then decide
how to route the traffic to the appropriate jail based on the SMTP or
HTTP protocol headers involved.  This is bog standard webserver stuff,
and I think it's not uncommon for mail servers either.

> I also read that it is possible to only run specific applications in a
> jail so the jail itself is not a completely new FreeBSD installation
> (see Handbook 15.3 Creating and Controlling Jails, first sentence). In
> case, I would have two jails and every jail's running a web server, now,
> there is a connection to IP X on port 80. Where is the connection going
> to? I guess this has to be configured at the jails' host acting as a
> gateway to the hosted jails and forwarding packages depending on the
> port that is used (e.g. 80 goes to Jail#1 and 8080 goes to Jail#2).

Yes, this is certainly possible.  The technique is called 'thin jails'
-- however, each jail will need a distinct IP, and the idea of the
jailed applications being able to bind to different ports on the same IP
doesn't work.  You can do fancy firewall redirects and stuff to make
this sort of thing work, but honestly, I think you'ld be better off
doing the proxying etc. at Layer 7 rather than Layer 3.

> I would like to understand this and the technical limitations better to
> get an idea about how many fixed public IP addresses I have to buy. So I
> can eventually save some money. :-)

Go with IPv6.  You'll have more IP numbers than you can possibly consume
thrust upon you...

> Thank you for any help. Sorry if I am asking for something that does not
> make any sense at all -- I am still busy trying to get the principles or
> options that exist in the set-up mentioned above.

	Cheers,

	Matthew


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 949 bytes
Desc: OpenPGP digital signature
URL: <http://lists.freebsd.org/pipermail/freebsd-questions/attachments/20140909/e791d744/attachment.sig>