10-STABLE and setfib

Fri Sep 5 07:37:24 UTC 2014

I rechecked it. Turned out it looks like it’s more cosmetic problem (?).

Let me explain what I wanted.

I wanted to take different routing table for a jail.

I created 2 same VMs on one Hyper-V hypervisor. Connected it to the
same virtual switch. Gave them same configuration with same IP
addresses. When I worked with one VM, another was turned off. And vise
versa. I used GENERIC kernel.

One of that machines was FreeBSD 10.0-p7 RELEASE amd64.

Another was fresh FreeBSD 10.1 r271152 PRERELEASE amd64.

VM has IP 192.168.8.14. An alias0 created for a jail with IP
192.168.8.13. The default router is 192.168.8.1.

Here is the same configs of that two VMs:

% cat /boot/loader.conf

autoboot_delay="2"

net.fibs=2

net.add_addr_allfibs=0

% cat /etc/rc.conf

hostname="10R" # 10S on 10.1-PRE

ifconfig_hn0="inet 192.168.8.14 netmask 255.255.255.0"

defaultrouter="192.168.8.1"

sshd_enable="YES"

ntpd_enable="YES"

ntpd_sync_on_start="YES"

dumpdev="AUTO"

ifconfig_hn0_alias0="inet 192.168.8.13/32 fib 1"

static_routes="fibnet fibdef"

route_fibnet="-net 192.168.8.0/24 -interface hn0 -fib 1"

route_fibdef="default 192.168.8.1 -fib 1"

jail_enable="YES"

jail_list="mailjail"

% cat /etc/jail.conf

exec.start = "/bin/sh /etc/rc";

exec.stop = "/bin/sh /etc/rc.shutdown";

exec.clean;

mount.devfs;

devfs_ruleset = 4;

$jailsdir = "/var/jails";

path = "$jailsdir/$name";

mailjail {

  mount.fdescfs;

  allow.sysvipc;

  allow.raw_sockets;

  allow.set_hostname;

  exec.fib = 1;

  ip4.addr = 192.168.8.13;

}

Here is the defference:

FreeBSD 10.0-p7 RELEASE amd64 (which is OK IMO)

boot message

...

hn0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500

        options=18<VLAN_MTU,VLAN_HWTAGGING>

        ether 00:15:5d:08:6f:0b

        inet 192.168.8.14 netmask 0xffffff00 broadcast 192.168.8.255

        inet6 fe80::215:5dff:fe08:6f0b%hn0 prefixlen 64 scopeid 0x2

        inet 192.168.8.13 netmask 0xffffffff broadcast 192.168.8.13

        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

        fib: 1

Starting devd.

add net 192.168.8.0: gateway hn0 fib 1

add net default: gateway 192.168.8.1 fib 1

add net default: gateway 192.168.8.1 fib 0

add net fe80::: gateway ::1 fib 0,1

add net ff02::: gateway ::1 fib 0,1

add net ::ffff:0.0.0.0: gateway ::1 fib 0,1

add net ::0.0.0.0: gateway ::1 fib 0,1

...

On host

% netstat -f inet -rn

Routing tables

Internet:

Destination        Gateway            Flags    Refs      Use  Netif Expire

default            192.168.8.1        UGS         0       91    hn0

127.0.0.1          link#1             UH          0        0    lo0

192.168.8.0/24     link#2             U           0      576    hn0

192.168.8.13       link#2             UHS         0       72    lo0 =>

192.168.8.13/32    link#2             U           0        0    hn0

192.168.8.14       link#2             UHS         0        0    lo0

On host

% setfib 1 netstat -f inet -rn

Routing tables (fib: 1)

Internet:

Destination        Gateway            Flags    Refs      Use  Netif Expire

default            192.168.8.1        UGS         0        5    hn0

192.168.8.0/24     00:15:5d:08:6f:0b  US          0        0    hn0

Inside the jail

% netstat -f inet -rn

Routing tables (fib: 1)

Internet:

Destination        Gateway            Flags    Refs      Use  Netif Expire

default            192.168.8.1        UGS         0        7    hn0

192.168.8.0/24     00:15:5d:08:6f:0b  US          0       78    hn0

FreeBSD 10.1 r271152 PRERELEASE amd64 (which is not OK IMO)

boot message

...

hn0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500

        options=18<VLAN_MTU,VLAN_HWTAGGING>

        ether 00:15:5d:08:6f:09

        inet 192.168.8.14 netmask 0xffffff00 broadcast 192.168.8.255

        inet6 fe80::215:5dff:fe08:6f09%hn0 prefixlen 64 scopeid 0x2

        inet 192.168.8.13 netmask 0xffffffff broadcast 192.168.8.13

        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

        fib: 1

Starting devd.

add net 192.168.8.0: gateway hn0 fib 1

add net default: gateway 192.168.8.1 fib 1

add net default: gateway 192.168.8.1 fib 0

route: writing to routing socket: Network is unreachable

add net fe80::: gateway ::1 fib 0

add net fe80::: gateway ::1 fib 1: Network is unreachable

route: writing to routing socket: Network is unreachable

add net ff02::: gateway ::1 fib 0

add net ff02::: gateway ::1 fib 1: Network is unreachable

route: writing to routing socket: Network is unreachable

add net ::ffff:0.0.0.0: gateway ::1 fib 0

add net ::ffff:0.0.0.0: gateway ::1 fib 1: Network is unreachable

route: writing to routing socket: Network is unreachable

add net ::0.0.0.0: gateway ::1 fib 0

add net ::0.0.0.0: gateway ::1 fib 1: Network is unreachable

...

On host

% netstat -f inet -rn

Routing tables

Internet:

Destination        Gateway            Flags    Netif Expire

default            192.168.8.1        UGS       hn0

127.0.0.1          link#1             UH        lo0

192.168.8.0/24     link#2             U         hn0

192.168.8.14       link#2             UHS       lo0

On host

% setfib 1 netstat -f inet -rn

Routing tables (fib: 1)

Internet:

Destination        Gateway            Flags    Netif Expire

default            192.168.8.1        UGS       hn0

192.168.8.0/24     00:15:5d:08:6f:09  US        hn0

192.168.8.13       link#2             UHS       lo0

192.168.8.13/32    link#2             U         hn0

Inside the jail:

% netstat -f inet -rn

Routing tables (fib: 1)

Internet:

Destination        Gateway            Flags    Netif Expire

192.168.8.13       link#2             UHS       lo0

So the difference between these two VMs:

-          dmesg messages like “Network is unreachable” on PRERELEASE.

-          Visibility of default router inside the jail on PRERELEASE.

At the same time it looks like this configuration works normally. I
can access the network from the jail on both systems.

So problem is more cosmetic (?).

2014-09-04 17:56 GMT+04:00 Pavel Timofeev <timp87 at gmail.com>:
> Hi!
> I've read this topic in forum
> https://forums.freebsd.org/viewtopic.php?f=7&t=47693 where described
> how to deal with default route for jail with different fib.
> I tried it on 2 same virt machines, but with different interface name
> and ip addresses than on forum.
> While it works in 10.0-RELEASE it doesn't work in 10.1-PRERELEASE r271030.
> It says 'Network is unreachable' while booting. Here is a piece of dmesg.boot:
>
>
> add net 172.16.220.0: gateway hn1 fib 1
> add net default: gateway 172.16.220.1 fib 1
> add net default: gateway 192.168.8.1 fib 0
> Additional inet routing options: gateway=YES.
> route: writing to routing socket: Network is unreachable
> add net fe80::: gateway ::1 fib 0
> add net fe80::: gateway ::1 fib 1: Network is unreachable
> route: writing to routing socket: Network is unreachable
> add net ff02::: gateway ::1 fib 0
> add net ff02::: gateway ::1 fib 1: Network is unreachable
> route: writing to routing socket: Network is unreachable
> add net ::ffff:0.0.0.0: gateway ::1 fib 0
> add net ::ffff:0.0.0.0: gateway ::1 fib 1: Network is unreachable
> route: writing to routing socket: Network is unreachable
> add net ::0.0.0.0: gateway ::1 fib 0
> add net ::0.0.0.0: gateway ::1 fib 1: Network is unreachable
>
>
> And 'netstat -rn' doesn't show routes inside the jail.
> Can anyone confirm such regression?