Source address selection failure when using RFC5549-style (v4-via-v6) routes

From: Felix <felix_at_ffetc.net>
Date: Tue, 05 Aug 2025 14:09:46 UTC
I've been experimenting a bit with routing IPv4 via IPv6 next-hops, 
something that FreeBSD has supported since 13.1
[https://reviews.freebsd.org/D30398].

My understanding (note: I'm not a professional network engineer) is that 
this is normally used with routing protocols like BGP (RFC5549, often 
called "extended nexthop") or Babel (RFC9229), where it's extremely 
convenient - the links between your routers only need link-local v6 
addresses to forward both v4 and v6 traffic, reducing configuration and 
saving IP addresses. Of course, each router should have /one/ routable 
v4 and v6 address so that it can return ICMP messages and so on, which 
one would typically assign on the loopback interface.

For experimentation, you can create these routes directly, without 
running one of these protocols, like so:

# route add -net 203.0.113.2/32 -inet6 fe80::2%bge2
add net 203.0.113.2: gateway fe80::2%bge2

# netstat -r4n
Routing tables

Internet:
Destination        Gateway            Flags         Netif Expire
...
203.0.113.2        fe80::2%bge2       UGHS           bge2
...

What I've discovered: trying to /originate/ a connection from a machine 
with such a route doesn't seem to work properly. Any attempts to open a 
TCP connection (e.g. ssh) will fail with EHOSTUNREACH:

# truss ssh 203.0.113.2
...
connect(3,{ AF_INET 203.0.113.2:22 },16)         ERR#65 'No route to host'
...
ssh: connect to host 203.0.113.2 port 22: No route to host
...
process exit, rval = 255

When choosing a source address for an outgoing connection, FreeBSD does 
a route lookup (which succeeds in this case), and as a comment in 
sys/netinet/in_pcb.c helpfully tells us:

	/*
	 * If the outgoing interface on the route found is not
	 * a loopback interface, use the address from that interface.
	 */

Because the outgoing interface doesn't /have/ a v4 address, a 
consistency check at the end of the function returns EHOSTUNREACH.

Linux also supports these v4-via-v6 routes, so I can compare how it 
behaves. Firstly, it will use the address specified in the `src` 
attribute of the route if there is one (FreeBSD, AFAIK, has no such 
notion). Otherwise, it follows some process to pick one of the machine's 
addresses to use as a source address, and will (if necessary, like in 
this scenario) pick an address from a different interface.

I'm not sure what the right thing for FreeBSD to do in this circumstance 
is. What do you think?

I also haven't tested whether this same issue affects the generation of 
ICMP responses (e.g. TTL expired, packet too big). If it does, that 
seems like much more of a concern for using FreeBSD on real routers.

Thanks,
- Felix