kern/189088: Assigning the same IP to multiple interfaces in different FIBs creates a host route for only one.

[Alan Somers]

>Number:         189088
>Category:       kern
>Synopsis:       Assigning the same IP to multiple interfaces in different FIBs creates a host route for only one.
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Mon Apr 28 22:30:00 UTC 2014
>Closed-Date:
>Last-Modified:
>Originator:     Alan Somers
>Release:        11.0 CURRENT
>Organization:
Spectra Logic
>Environment:
FreeBSD alans-fbsd-head 11.0-CURRENT FreeBSD 11.0-CURRENT #55 r265061M: Mon Apr 28 15:08:01 MDT 2014     alans at ns1.eng.sldomain.com:/vmpool/obj/usr/home/alans/freebsd/head/sys/GENERIC  amd64
>Description:
Assigning the same IP to multiple interfaces simultaneously sounds insane, but some people do it.  However, if you assign them to separate FIBs as well as separate interfaces, bad stuff happens.  For one thing, only the first FIB used will get a host route.  For another, if you delete the addresses in FIFO order the host route will never get deleted (though it will if you delete them in LIFO order).
>How-To-Repeat:
On a system with net.add_addr_allfibs=0 and net.fibs=4, Run the following commands:

# ifconfig tap0 create
# ifconfig tap1 create
# ifconfig tap0 192.0.2.2/32 fib 2
# ifconfig tap1 192.0.2.2/32 fib 3
# setfib 2 netstat -rn -f inet
Routing tables (fib: 2)

Internet:
Destination        Gateway            Flags    Netif Expire
192.0.2.2          link#3             UHS       lo0
192.0.2.2/32       link#3             U        tap0
# setfib 3 netstat -rn -f inet
Routing tables (fib: 3)

Internet:
Destination        Gateway            Flags    Netif Expire
192.0.2.2/32       link#4             U        tap1


Notice that FIB 3 does not get a 192.0.2.2 host route.  Now try deleting the addresses in FIFO order:


# ifconfig tap0 -alias 192.0.2.2
# ifconfig tap1 -alias 192.0.2.2
# setfib 2 netstat -rn -f inet
Routing tables (fib: 2)

Internet:
Destination        Gateway            Flags    Netif Expire
192.0.2.2          link#4             UHS       lo0
>Fix:


>Release-Note:
>Audit-Trail:
>Unformatted: