Re: IPv6 inconsistent local routing

On Wed, Oct 20, 2021 at 10:06:00PM +0000, Bjoern A. Zeeb wrote:

! > You might say, it does not usually make sense to route traffic local-to-
! > local over an IPadress of an outbound interface. But actually this
! > is common practice in e.g. failover scenarios, where you don't know
! > in the app configs if your address is currently local or not - you just
! > know it carries the service.
! 
! 
! There used to be:
! 
! $ sysctl -d net.link.ether.inet.useloopback net.inet6.icmp6.nd6_useloopback
! net.link.ether.inet.useloopback: Use the loopback interface for local
! traffic
! net.inet6.icmp6.nd6_useloopback:
! 
! which defined this bahviour.
! 
! These were removed in/after the FreeBSD 10 times I think?
! I’d start looking from there.

Hi, thank you, yeah I found these. While pointing someway into that
rabbithole, it is actually a dead end at first: because, these options
influence the routing table, and they are now default-ON anyway.

Also, in my case, the routing table is correctly populated with lo0,
so this is not the problem. And, the routing table only considers
OUTgoing packets, and as we can see in my message above, my OUTgoing
packets are all fine and go via lo0.

It is the INcoming path that create this problem. Glancing further
thru the code, I find there is a "m_pkthdr.rcvif" variable from which
ipfw gets the incoming-interface info. But then that variable is
spread all over the whole networking code and gets used everywhere.
So unless one does already dwell in the networking code and kernel
debugger, this will be extensive labour to figure out. 

But, I just found, the other way is just as ugly: when I move my stuff
to work via localhost, I need, among other things, also new kerberos
principals, because these then must read e.g. "postgres/localhost",
and must provide the correct principal to all concerned places. It's
alot of effort, and it's the wrong direction.

Normally, people thow their lamp-stack (or whatever) onto one machine,
configure it all as localhost, and everything works, and nothing is
modular and nothing is extensible. And then when they notice there is
a problem they have no clue and pay a lot of money for containers.
I started the other way round, always and only use the external
interface names (and don't mix these with the hostname, the hostname
is something entirely different and only relevant for mail, if at
all), configure everything onto these names, and then everything works
from everywhere right from the beginning.
Except when ipfw gets these creepily mixed interface
tags. Ob-rant. Thanks for listening.

Cheerio,
PMc