Source address selection failure when using RFC5549-style (v4-via-v6) routes
Date: Tue, 05 Aug 2025 14:09:46 UTC
I've been experimenting a bit with routing IPv4 via IPv6 next-hops,
something that FreeBSD has supported since 13.1
[https://reviews.freebsd.org/D30398].
My understanding (note: I'm not a professional network engineer) is that
this is normally used with routing protocols like BGP (RFC5549, often
called "extended nexthop") or Babel (RFC9229), where it's extremely
convenient - the links between your routers only need link-local v6
addresses to forward both v4 and v6 traffic, reducing configuration and
saving IP addresses. Of course, each router should have /one/ routable
v4 and v6 address so that it can return ICMP messages and so on, which
one would typically assign on the loopback interface.
For experimentation, you can create these routes directly, without
running one of these protocols, like so:
# route add -net 203.0.113.2/32 -inet6 fe80::2%bge2
add net 203.0.113.2: gateway fe80::2%bge2
# netstat -r4n
Routing tables
Internet:
Destination Gateway Flags Netif Expire
...
203.0.113.2 fe80::2%bge2 UGHS bge2
...
What I've discovered: trying to /originate/ a connection from a machine
with such a route doesn't seem to work properly. Any attempts to open a
TCP connection (e.g. ssh) will fail with EHOSTUNREACH:
# truss ssh 203.0.113.2
...
connect(3,{ AF_INET 203.0.113.2:22 },16) ERR#65 'No route to host'
...
ssh: connect to host 203.0.113.2 port 22: No route to host
...
process exit, rval = 255
When choosing a source address for an outgoing connection, FreeBSD does
a route lookup (which succeeds in this case), and as a comment in
sys/netinet/in_pcb.c helpfully tells us:
/*
* If the outgoing interface on the route found is not
* a loopback interface, use the address from that interface.
*/
Because the outgoing interface doesn't /have/ a v4 address, a
consistency check at the end of the function returns EHOSTUNREACH.
Linux also supports these v4-via-v6 routes, so I can compare how it
behaves. Firstly, it will use the address specified in the `src`
attribute of the route if there is one (FreeBSD, AFAIK, has no such
notion). Otherwise, it follows some process to pick one of the machine's
addresses to use as a source address, and will (if necessary, like in
this scenario) pick an address from a different interface.
I'm not sure what the right thing for FreeBSD to do in this circumstance
is. What do you think?
I also haven't tested whether this same issue affects the generation of
ICMP responses (e.g. TTL expired, packet too big). If it does, that
seems like much more of a concern for using FreeBSD on real routers.
Thanks,
- Felix