FreeBSD jail can't talk to internet through multiple routers

[Andreas Nilsson]

On Wed, May 29, 2013 at 4:06 AM, Julian Elischer <julian at freebsd.org> wrote:

> On 5/29/13 7:57 AM, Jeff wrote:
>
>> Hi,
>>
>> I run PCBSD 9.1 and have a jail setup (uses the Warden PBI to set it up).
>>
>> In that jail which has it's own local IP like 192.168.1.12, I have an
>> Apache server running Drupal.
>>
>> Normally when I connect the computer to a single router that is connected
>> to a modem, I set "nameserver 192.168.1.1", i.e. the router LAN IP or
>> gateway, in etc/resolv.conf and have no problems.
>>
>> Now I have added a 2nd router daisy chained from the primary router,
>> running a subnet (primary router has IP: 192.168.1.1 and secondary router:
>> 192.168.2.1).
>>
>> The computer running the jail is plugged into the secondary router.
>>
>> The problem is, the jail can't contact the internet.  I can SSH into the
>> jail but it takes a very long time to connect, like 30 seconds or so.
>>
>>
>> I've tried different IP addresses for "nameserver" but nothing works.
>>
>> I have no problems using the internet from the main part of the computer,
>> just the jails.
>>
>>
>> Any ideas why this happens and how to get around it?  I've had this
>> problem for years with different versions of FreeBSD.
>>
>> Do I need to create a static route through to the gateway, and if so, why
>> is that not a problem using a browser from the main part of the machine?
>>
>
>
> basically your jail is using the same routing as the rest of the machine
> you have several options, though they may not all be supported in the
> PCBSD 9.1 jail system
>
> 1/ you could use ipfw to do packet forwarding
>   this is what we used to before we had #2 and #3.
> 2/ you can specify that the jail should use a different FIB (routing table)
>   you should look up setfib(1) and setfib(2) and follow the 'see also'
> pointers as well.
> 3/ you can use VIMAGE and set up a jail with a completely separate network
> stack.
> DOcumentation for this is a bit hard to find but use the 'vnet' option in
> jail(8)
> nad look up VIMAGE and vnet  in google.
>
>
>
>>
>> Thanks,
>>
>> Jeff
>> ______________________________**_________________
>> freebsd-net at freebsd.org mailing list
>> http://lists.freebsd.org/**mailman/listinfo/freebsd-net<http://lists.freebsd.org/mailman/listinfo/freebsd-net>
>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@**freebsd.org<freebsd-net-unsubscribe at freebsd.org>
>> "
>>
>>
>>
> ______________________________**_________________
> freebsd-net at freebsd.org mailing list
> http://lists.freebsd.org/**mailman/listinfo/freebsd-net<http://lists.freebsd.org/mailman/listinfo/freebsd-net>
> To unsubscribe, send any mail to "freebsd-net-unsubscribe@**freebsd.org<freebsd-net-unsubscribe at freebsd.org>
> "
>


>From the example ips I take it you are behind "double nat", not just behind
2 routers? When you say "jail cant connect to the internet", is it just dns
querys that fails, or ip connectivity? Are you running any firewall on the
host?

 I was forced to used a similar setup for a while, but I never saw those
problems. Timeouts on ssh could point to dns failures: did you update the
resolv.conf in the jails as well as on the host?

If you start the jail with allow.raw_sockets enabled ( you didn't mention
what method, ie rc.conf jail.conf or just jail -c, you use to start the
jails so use the appropriate method of passing that arg), could you then
from within the jail do some pings and digs like:
dig @8.8.8.8 freebsd.org
dig @192.168.2.1 freebsd.org
dig @192.168.1.1 freebsd.org
ping 8.8.8.8
ping 192.168.2.1
ping 192.168.1.1

Best regards
Andreas