Re: Interface routes and multiple fibs

> On Oct 18, 2022, at 15:16, Paul Procacci <pprocacci@gmail.com> wrote:
> 
> 
> 
> On Tue, Oct 18, 2022 at 5:12 PM Dan Mahoney <freebsd@gushi.org <mailto:freebsd@gushi.org>> wrote:
> All,
> 
> Maybe a question for the -net or -rc people.  If I should ask there, let me know.
> 
> I'm running with multiple fibs.  One fib is just box management, ssh, etc.  The other fib (which takes BGP routes from peers via BIRD) does DNS anycast things.  The DNS server runs in fib 1.  Our default route is added to both fibs.
> 
> My fib0 routing table looks like this:
> 
> Internet:
> Destination        Gateway            Flags     Netif Expire
> default            192.159.249.233    UGS        bge0
> 127.0.0.1          link#5             UH          lo0
> 182.159.249.232/29 <http://182.159.249.232/29> link#1             U          bge0
> 182.159.249.236    link#1             UHS         lo0
> 
> Fib 1 is missing that final route.:
> 
> default            182.159.249.233    UGS        bge0
> 127.0.0.1          link#5             UH          lo0
> 182.159.249.232/29 <http://182.159.249.232/29> link#1             U          bge0
> 
> I've noticed that when I try to do a query (with dig) against it from fib 0, it sends over lo0 to the named process, but the reply packet just gets sent out ON BGE0, and is never received, since dig is listening on the interface it sent the packet over (lo0) to hear the response, which, near as I can tell with tcpdump -i bge0, just goes out on the wire
> 
> Obviously, we can add the static route to that second fib with:  route add -host 182.159.249.236 -interface lo0 -fib 1.
> 
> Yes, we can also make this stick useing default_routes in rc.conf.
> 
> But it feels like we shouldn't have to.  This feels like a glitch, and that if all fibs get the SUBNET route , they should also get the loopback.
> 
> -Dan
> 
> 
> Why would you not expect to add a route for it?
> The same subnets can exist in different fibs and be part of different lan segments a la vlans.  Routes are required.

But...the same route is added for the SUBNET on both fibs automatically, even though bge0 is in fib 1.  Just not for the actual host.  This feels woefully inconsistent.