setfib (ez)jails and wierd routing

[Marek Zarychta]

On Mon, Oct 16, 2017 at 04:22:04PM +0200, Marko Cupać wrote:
> Hi,
> 
> I have already asked this on -jail two weeks ago, but perhaps this is
> better place to ask.
> 
> I notice wierd routing in my setfib (ez)jails setup.
> 
> I have a server with multiple NICs. setfib should ensure that LAN jails
> (setfib 1) can not talk to DMZ jails (setfib 2) over loopbacks, but
> need to go through firewalls as though they were physical boxes.
> 
> pacija at warden3:~ % sudo setfib 1 netstat -rn
> Routing tables (fib: 1)
> 
> Internet:
> Destination        Gateway            Flags     Netif Expire
> default            10.30.19.190       UGS        bce0
> 10.30.19.160/27    00:1c:c4:de:0a:86  US         bce0
> 127.0.0.1          lo0                UHS         lo0
> 127.0.1.0/24       lo1                US          lo1
> 
> pacija at warden3:~ % sudo setfib 2 netstat -rn
> Routing tables (fib: 2)
> 
> Internet:
> Destination        Gateway            Flags     Netif Expire
> default            193.53.106.254     UGS        bce1
> 127.0.0.1          lo0                UHS         lo0
> 127.0.2.0/24       lo2                US          lo2
> 193.53.106.0/24    00:1c:c4:de:0a:84  US         bce1
> 
> Host has the same default route as fib 1:
> 
> pacija at warden3:~ % sudo netstat -rn
> Routing tables
> 
> Internet:
> Destination        Gateway            Flags     Netif Expire
> default            10.30.19.190       UGS        bce0
> ...
> 
> If I ssh from the Internet into DMZ jail, everything works as expected.
> But if I ping DMZ jail from the Internet, I see reply packets leaving
> not the interface they came from (bce1, public address space, DMZ), but
> another one (bce0, private address space, LAN). This is kinda
> understandable, because jail on fib2 does not have ICMP enabled, so
> it is not DMZ jail, but the host (which is in fib 0) who replies to
> packets via its default gateway (router on a private LAN).
> 
> Is there an easy and elegant way to solve this? Like binding IP address
> to fib? I wouldn't like to have to fire up pf on host and meddle with
> reply-to rules in order to achieve this, I'd rather revert to old setup
> of separate physical servers for each network.
> 
Hi,

try after to set "ifconfig bce1 fib 2" after disabling PF. 
This  should do the work.


-- 
Marek Zarychta
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: not available
URL: <http://lists.freebsd.org/pipermail/freebsd-net/attachments/20171016/0e91129b/attachment.sig>