setfib, jails and loopback interfaces

[Marko Cupać]

Hi,

I'm not subscribed to the list, could you please keep me in CC?

I'm using ezjail as instructed in Handbook, assigning jails
lo1|127.0.0.X,bce0|10.66.66.X addresses, in order to keep jails'
loopback traffic off host's, and in order to be able to keep internal
services on lo1 (such as redis, mongodb, mysql etc.), and external on
bce0 (such as apache, unifi5 etc.).

Recently I got a server with multiple NICs, and I'd like to serve both
LAN and DMZ services from it. I found some information on how to
accomplish that with setfib:

# cat /boot/loader.conf
net.fibs=4
net.add_addr_allfibs=0

# cat /etc/rc.conf
...
cloned_interfaces="lo1"
static_routes="nix nixd"
route_nix="-net 10.66.66.0/24 -interface bce0 -fib 1"
route_nixd="default 10.66.66.254 -fib 1"
...

In this setup, services bound to bce0 interface work fine, but they
can't contact internal services on lo1. I guess it has something to do
with jail routing, but can't figure out what.

Thank you in advance for any hints.
-- 
Before enlightenment - chop wood, draw water.
After  enlightenment - chop wood, draw water.

Marko Cupać
https://www.mimar.rs/