ZFS, Jails, network, routing, domains and IP addresses

[markham_breitbach at ssimicro.com]

On 2014-09-09, 7:22 AM, Niklaas Baudet von Gersdorff wrote:
> Hi,
>
> I am not an educated computer scientist but got in touch with UNIX and
> Linux quite early. Since then I ran several servers and am somehow
> finding my way through the IT world by readings lots of blogs,
> articles and mailing lists about the topic as a hobby. At the moment I
> am running a root server at some provider who I don't like anymore
> (this has its reasons) and would like to switch the provider. Because
> this will be some work in any event I thought about simultaneously
> switching from Debian to FreeBSD since this is something I would like
> to do for quite some time.
>
> The main reasons for switching to FreeBSD are jails and the ports
> system. My question concerns jails and the set-up I thought about. If
> you have any thoughts about it please just give me some short hints
> and I'll be very happy about that. :-)
>
> So, the future server has 48 GB of RAM and 2 2TB HDDs. I thought about
> installing FreeBSD 10 with ZFS (on /) mirroring both HDDs. (I already
> did that set-up smaller and virtualized on my desktop machine and that
> worked great.) I would like to use jails since I've got several
> domains to administer and each domain belongs to another friend of
> mine. Hence, they should not get any access to the jail host or other
> jail clients. So, I would like to use jails to virtualize several
> servers. On every host there'll be a Postfix and Apache installation.
> So, everything stays quite simple. Nothing complex.
Jails are a great solution for this.  I have used jails to roll out
identical systems across a large geographic area, so that each jail uses
a local configuration file to set some local variables (IP address,
hostname, etc.) that are inserted at jail startup time.  In this way, I
can maintain a single "image" that can be deployed and installed to
quickly upgrade dozens of servers.  Another trick I have used is to
setup a base "instance" that is nullfs mounted for individual jails
which things like /var and /home remain unique and RW.  I am currently
running about 100 apache AMP hosts this way, so I can quickly upgrade
apache once in the single instance and all 100 hosts are instantly upgraded.
>
> 1. ZFS and Jails
>
> It would be cool if I could simplify the process of updating the
> software that is running in every jail. I searched in the web for some
> information and also had a look at the FreeBSD mailing lists. It looks
> like it's quite a popular set-up to create a "base" FreeBSD Jail that
> is cloned with the help of ZFS if there is a new jail needed. The
> ports tree is mounted with a nullfs in every jail so updating the
> "main" ports tree would lead to the software in every jail getting
> updated. Or am I understanding something totally wrongly here?
>
> While reading I also got the impression there are different methods
> for maintaining Jails with ZFS. I would be very thankful if anyone
> will point out the different approaches that exist (some articles on
> the net seem outdated). Maybe a quick reference to necessary man pages
> are already enough, then I can do further research on my own. :-)
>
> 2. Jails and routing
>
> The main question is: Is it possible for the jails' host to
> distinguish between incoming connections depending on the domain
> look-up they did? If it is possible I would like to use as less IP
> addresses as possible. Could be that it's technically not possible at
> all but I thought there is maybe some way to do it and someone knows.
> The idea is the jails' host does something like this: Connection to
> Domain#1 established so everything goes to Jail#1, Connection to
> Domain#2 established so everything foes to Jail#2, ... but the jails
> and the jails' host use the same IP X.
Technically what you are asking is not possible.  The server doesn't
know what lookup the client has done, although http can send a header to
identify the domain requested.  This will allow you to proxy requests to
the appropriate jail, but I suspect the money you save in IP addresses
will quickly be lost double in support time if you try to
over-complicate things with layer7 proxies and clever routing tricks,
double again when something goes wrong.
>
> I also read that it is possible to only run specific applications in a
> jail so the jail itself is not a completely new FreeBSD installation
> (see Handbook 15.3 Creating and Controlling Jails, first sentence). In
> case, I would have two jails and every jail's running a web server,
> now, there is a connection to IP X on port 80. Where is the connection
> going to? I guess this has to be configured at the jails' host acting
> as a gateway to the hosted jails and forwarding packages depending on
> the port that is used (e.g. 80 goes to Jail#1 and 8080 goes to Jail#2).
>
> I would like to understand this and the technical limitations better
> to get an idea about how many fixed public IP addresses I have to buy.
> So I can eventually save some money. :-)
> Thank you for any help. Sorry if I am asking for something that does
> not make any sense at all -- I am still busy trying to get the
> principles or options that exist in the set-up mentioned above.
>
> Best regards,
>