exec.fib and a jail in two subnets

[Grzegorz Junka]

Hi,

I am not sure if this question fits better to net or jail list so please 
delete one crosspost when replying.

I have two routers in separate subnets (say 10.0.0.0/16 and 
172.16.0.0/16). I have enabled multiple fibs on the host and I am trying 
to setup a jail so that packets from one router are returned to the same 
router. The second subnet is configured like this:

setfib 1 route add -net 172.16.0.0/16 -iface lagg0
setfib 1 route add default 172.16.0.1

When the jail configuration is (differences in red):

mta {
   exec.fib=1;
   ip4.addr = 172.16.0.2;
   interface = lagg0;
}

router 172.16.0.1 is able to send to and receive packets from the jail 
as expected.

When the jail configuration is:

mta {
   ip4.addr = 10.0.0.2,172.16.0.2;
   interface = lagg0;
}

then router 10.0.0.1 is also able to send and receive packets from the 
jail as expected.

However, when the configuration is:

mta {
exec.fib=1;
   ip4.addr = 10.0.0.2,172.16.0.2;
   interface = lagg0;
}

then router 172.16.0.1 is no longer able to receive a response from the 
jail. The router's event log shows entry similar to the following two 
about 2 minutes apart:

IN: ACCEPT [54] Connection opened (Port Forwarding: TCP [172.16.0.2]:80 
<-​-​> [212.159.95.213]:80 -​ -​ -​ [111.202.101.2]:34172 
CLOSED/SYN_SENT ppp3 NAPT)
IN: ACCEPT [57] Connection closed (Port Forwarding: TCP [172.16.0.2]:80 
<-​-​> [212.159.95.213]:80 -​ -​ -​ [111.202.101.2]:34172 
CLOSED/SYN_SENT ppp3 NAPT)

My question is why the 10.0.0.1 router is able to communicate with the 
jail in the second configuration but 172.16.0.1 is not able to 
communicate with the jail in the third configuration. Is it because of 
order of IPs in ip4.addr?

When the jail is started jls shows only the first IP from either of the 
configuration list above (i.e. 10.0.0.2 even if exec.fib is set to 1). 
So my guess is that the first IP is somehow a default IP?

Then my additional question is if it's possible for a jail to be in two 
subnets at the same time, i.e. so that when the jail responds to a 
packet received from router 10.0.0.1 it sends it to the default route 
from fib0 and when it responds to a packet received from 172.16.0.1 it 
sends it to the default route from fib1. What exec.fib should be in such 
a case?

Any help would be greatly appreciated. Thanks!

GrzegorzJ