Re: ipv4 route with ipv6 local link nexthop ?

> On Mar 13, 2024, at 9:09 PM, Zhenlei Huang <zlei@FreeBSD.org> wrote:
> 
> 
> 
>> On Mar 13, 2024, at 12:19 PM, Zhenlei Huang <zlei@FreeBSD.org <mailto:zlei@FreeBSD.org>> wrote:
>> 
>> 
>> 
>>> On Mar 13, 2024, at 4:36 AM, Benoit Chesneau <benoitc@enki-multimedia.eu <mailto:benoitc@enki-multimedia.eu>> wrote:
>>> 
>>> On latest freebsd 14.0 release , I am trying to join an ipv4 address over an IPV6 local link using the following command :
>>> 
>>> `route add -net 10.200.1.1/32 -inet6 fe80::9439:36ff:fef0:7cbd%vlan200`
>>> 
>>> But I  get the following error:
>>> 
>>> ```
>>> route add -net 10.200.1.1/32 -inet6 fe80::9439:36ff:fef0:7cbd%vlan200
>>> add net 10.200.1.1: gateway fe80::9439:36ff:fef0:7cbd%vlan200 fib 0: Invalid argument
>> 
>> I guess that is regression by transition of route(8) to NETLINK.
>> 
>> Can you please build sbin/route from source without NETLINK and give it another try ?
>> 
>> ```
>> # cd /usr/src/sbin/route
>> # setenv MK_NETLINK_SUPPORT no
>> # make
>> ```
> 
> Hi Benoit,
> 
> I managed to repeat this bug.
> 
> As a workaround, add an IPv4 address to the outgoing interface (vlan200 in your case),
> ```
> # ifconfig vlan200 inet x.x.x.x/y
> ```
> 
> then retry adding the route
> ```
> # route add -net 10.200.1.1/32 -inet6 fe80::9439:36ff:fef0:7cbd%vlan200
> ```
> 

Hi Alexander,

I think the logic of adding route from NETLINK routine is not complete.

After bumping netlink debug level,
```
# sysctl net.netlink.debug.nl_route_debug_level=9
```

I got these from kernel message:
```
# dmesg | tail
...
[nl_route] finalize_nhop: Unable to determine ifa, skipping
[nl_route] rtnl_handle_newroute: Error creating nexthop
...
```

From https://cgit.freebsd.org/src/tree/sys/netlink/route/rt.c?h=releng/14.0#n753 :

```
	/* Both nh_ifp and gateway are set */
	if (nh->nh_ifa == NULL) {
		const struct sockaddr *gw_sa = &nh->gw_sa;

		if (gw_sa->sa_family != dst->sa_family) {
			/*
			 * Use dst as the target for determining the default
			 * preferred ifa IF
			 * 1) the gateway is link-level (e.g. direct route)
			 * 2) the gateway family is different (e.g. IPv4 over IPv6).
			 */
			gw_sa = dst;
		}

		struct ifaddr *ifa = ifaof_ifpforaddr(gw_sa, nh->nh_ifp);
		if (ifa == NULL) {
			NL_LOG(LOG_DEBUG, "Unable to determine ifa, skipping");
			*perror = EINVAL;
			return (NULL);
		}
		nhop_set_src(nh, ifa);
	}
```
If the outgoing interface `nh->nh_ifp` does not have any IPv4 addresses, we should try ifaddrs on other interfaces,
and have 0.0.0.0 as the last sort ( in case the net stack is IPv6 only, i.e., no IPv4 addresses available )


>> 
>>> ```
>>> 
>>> ping6 to `fe80::9439:36ff:fef0:7cbd%vlan200` works. What I am doing wrong? Configuration of the interface is the following:
>>> 
>>> ```
>>> vlan200: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
>>>         options=4600703<RXCSUM,TXCSUM,TSO4,TSO6,LRO,RXCSUM_IPV6,TXCSUM_IPV6,MEXTPG>
>>>         ether fe:d9:dd:2d:35:c5
>>>         inet6 fe80::fcd9:ddff:fe2d:35c5%vlan200 prefixlen 64 scopeid 0x5
>>>         groups: vlan
>>>         vlan: 200 vlanproto: 802.1q vlanpcp: 0 parent interface: iavf0
>>>         media: Ethernet autoselect (10Gbase-SR <full-duplex>)
>>>         status: active
>>>         nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL
>>> ```
>>> 
>>> 
>>> 
>>> Benoît 
>> 
>> Best regards,
>> Zhenlei