Jails and IP Aliasing

[David Allen]

On Tue, Jul 8, 2008 at 2:24 AM, Mel <fbsd.questions at rachie.is-a-geek.net> wrote:
> On Monday 07 July 2008 18:51:33 David Allen wrote:
>
>> Granted, everything is really happening over the loopback address, but a
>> connection originating from the jailhost to a jail should appear to be
>> using the jailhost's IP address, or so I'd like to think.  If it doesn't,
>> then the scenario is awkward at best when trying to understand or debug
>> issues.
>
> To debug this, you need to 'add jail support to sockstat'. This sounds hard,
> and it is, but you can fake it, since sockstat gives you the PID. With a
> little creative scripting, you can call `ps -o state' for each PID in the
> list, look for the capital 'J' and if it is, add the 'J' to the line.

Been there and done that.  When I first stated working with jails, I
discovered that most standard utilities didn't offer any support for
jails, and chaining commands got to be really old fast.   I ended up
writing a few Perl scripts and routinely use those instead.  IIRC,
there's a jail-related port that offers a collection of something
similar.

Still, we're talking about a very limited subset of tools and
functionality.  What about tcpdump?  Or firewall rules?  Or any other
network tool?

There was a post recently (Matthew Seaman's name comes to mind) that
suggested binding jails to addresses in the loopback range and then
using firewall rules to redirect the traffic accordingly.  There's a
possibility that may help in this case, but that layer of added
complexity isn't much of an improvement over seeing connections with
seemingly identical endpoints and interpreting the results in my head.

>> The thought occurred to me, however, that I could add a new network card
>> and reserve that for the IP aliases needed by the jails.  But I'm not sure
>> whether that will work in telling me who's who, or whether I'll discover
>> another gotcha.  ;-)
>
> It will add more gotcha's, unless you put each network card in a different
> network. With the IP's given here, you tell the host that 10.0.1.0/24 is on
> fxp0, so it will never go to fxp1 for 10.0.1.4.

You're probably right.  I'm wondering, though, if by moving the jails
into their own network space and adding routing into the mix, the end
result may be more satisfactory?

Setting aside the fun of mental gymnastics, the conclusion seems to be
don't run anything on the jail host that would initiate a connection
to a service running inside a jail.  Unless, of course, you don't mind
being confused (at least from a networking perspective) by WTF you're
seeing.  ;-)

Either way, thanks very much for the input.