fib issue with jails.

[Julian Elischer]

On 3/2/15 12:12 AM, Loïc Blot wrote:
> Hello,
> i'm trying to implement jails over multiples networks, using VLANs, with different default routes. The network stack is simple
>
> igb0-3 into lagg0
> vlan 10-30 over lagg0
> jails over VLANs using a fib for each VLAN (but no fib set on the VLAN iface itself)
>
> Whereas it works for a week on my server, after a reboot, the outgoing packets aren't routed to lagg and then outgoing requests doesn't work (like DNS requests), i don't find why.
>
> The fib is correctly set
>
> /etc/rc.local:
> setfib 1 route add -net 192.168.136.0/24 -iface vlan136
> setfib 1 route add default 192.168.136.254
>
> root at jh1:~ # setfib 1 netstat -rnfinet
> Routing tables (fib: 1)
>
> Internet:
> Destination        Gateway            Flags      Netif Expire
> default            192.168.136.254    UGS     vlan136
> 192.168.136.0/24   ac:16:2d:96:e5:04  US      vlan136
>
> and the jails are correctly configured:
>
> root at jh1:~ # cat /var/run/jail.idevmysql.conf
> # Generated by rc.d/jail at 2015-02-27 10:38:05
> devmysql {
>      host.hostname = "devmysql.local.net";
>      path = "/jails/dev/devmysql";
>      ip4.addr += "vlan136|192.168.136.50/32";
>      exec.fib = "1";
>      allow.raw_sockets = 0;
>      exec.clean;
>      exec.system_user = "root";
>      exec.jail_user = "root";
>      exec.start += "/bin/sh /etc/rc";
>      exec.stop = "";
>      exec.consolelog = "/var/log/jail_idevmysql_console.log";
>      mount.fstab = "/etc/fstab.idevmysql";
>      mount.devfs;
>      mount.fdescfs;
>      mount +=  "procfs /jails/dev/idevmysql/proc procfs rw 0 0";
>      allow.mount;
>      allow.set_hostname = 0;
>      allow.sysvipc = 0;
> }
>
> Routing is also enabled:
>
> root at jh1:~ # sysctl net.inet.ip.forwarding
> net.inet.ip.forwarding: 1
>
> If we are trying to contact the jail from an external host, for example with ansible, the SSH connection works very well but it seems outgoing initiated connections are staying on vlan136 but not forwarded to lagg0.
> Have you got any idea ?

Can you explain in more depth, what you mean by that last bit?
"staying on vlan136 but not forwarded to lagg0" .
I am not sure how you come to this idea and what you mean by it.


have you considered if you could use VIMAGE/VNET based jails?

>
> Thanks in advance
> Regards,
>
> Loïc Blot,
> UNIX Systems, Network and Security Engineer
> http://www.unix-experience.fr (http://www.unix-experience.fr)
> _______________________________________________
> freebsd-net at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
>
>
>