IPv6 routes leaking between FIBs?

[Jason Healy]

Hello,

Trying out FreeBSD for the first time to build a firewall box that’s multi-core and runs PF.  I’m very interested in the FIB code, as it lines up well with the way my core networking equipment works and should allow me to route traffic on an interface that’s logically separate from the management interfaces.

I’ve been playing for a bit with the FIB features, but I’m getting hung up on IPv6.  I’m trying to set up two interfaces on my box to each have a different FIB, and to not leak routes between the interfaces:

# sysctl net.add_addr_allfibs=0
# ifconfig em1 inet 192.0.2.1/24 fib 1
# ifconfig em1 inet6 2001:db8:dead:beef::1/64 fib 1
# ifconfig em2 inet 203.0.113.1/24 fib 2
# ifconfig em2 inet6 2001:db8:cafe:babe::1/64 fib 2

If I then check the routing tables for each FIB, here’s what I get:

# setfib -F 1 netstat -rn

Routing tables (fib: 1)

Internet:
Destination        Gateway            Flags      Netif Expire
192.0.2.0/24       link#2             U           em1
192.0.2.1          link#2             UHS         lo0

Internet6:
Destination                       Gateway                       Flags      Netif Expire
2001:db8:cafe:babe::/64           link#3                        U           em2
2001:db8:dead:beef::/64           link#2                        U           em1
2001:db8:dead:beef::1             link#2                        UHS         lo0
fe80::%em1/64                     link#2                        U           em1
fe80::a00:27ff:fef6:162a%em1      link#2                        UHS         lo0
fe80::%em2/64                     link#3                        U           em2
fe80::%lo0/64                     link#5                        U           lo0


# setfib -F 2 netstat -rn

Routing tables (fib: 2)

Internet:
Destination        Gateway            Flags      Netif Expire
203.0.113.0/24     link#3             U           em2
203.0.113.1        link#3             UHS         lo0

Internet6:
Destination                       Gateway                       Flags      Netif Expire
2001:db8:cafe:babe::/64           link#3                        U           em2
2001:db8:cafe:babe::1             link#3                        UHS         lo0
2001:db8:dead:beef::/64           link#2                        U           em1
fe80::%em1/64                     link#2                        U           em1
fe80::%em2/64                     link#3                        U           em2
fe80::a00:27ff:fe62:d267%em2      link#3                        UHS         lo0
fe80::%lo0/64                     link#5                        U           lo0


Note that as expected, the IPv4 routes are constrained to their FIB (192.0.2.0 to FIB 1 and 203.0.113.0 to FIB 2).  However, the IPv6 routes (deadbeef and cafebabe) leak between the FIBs; both prefixes that I add are listed in both FIBs (as well as the link-local stuff).

According to:

  https://www.freebsd.org/news/status/report-2012-01-2012-03.html#Multi-FIB:-IPv6-Support-and-Other-Enhancements

IPv6 parity is claimed for the FIB code, so I’m not sure if I’m doing it wrong, or if there’s a problem with the FIB code and IPv6 routes.

Thanks in advance for any help or clarification!

Jason