From nobody Wed Mar 13 13:17:32 2024 X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TvrfC5M3Hz5CsCD for ; Wed, 13 Mar 2024 13:17:39 +0000 (UTC) (envelope-from zlei@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TvrfC4h7vz5284; Wed, 13 Mar 2024 13:17:39 +0000 (UTC) (envelope-from zlei@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1710335859; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=vkt9PPHLo9GRjskLYlv+otGkV2CAA6QQKHyE8/QGnSI=; b=aC1g883C33GbYiZFhoYEyCM+6t4FENpwItwo6OhGkRsd7YKgSfEWlRluUAT3Xu/Rfztlk2 gB7g8TiS1OHXCLwGZJKMNCBYpe/caSSv0R6Bxb5YAyF8yC64TZ0uhaMlDNNc04FhPBCu9q oUUtzOdAZBWQ1S0lzsFyrHllL/R2zc7aY+iJJEyB9jhPPaRElrWfTFYeR70pZyoJaIYqEJ 9q8ENPRnqz3i/eSahSH9qwuGomefedmadD1dnqfcWT2iE1gMkAC9BnBZ4A4P+aEvJmOgQU rlDzxNbLz8zLNB0rLoixVKxmk0PVWG67tlO/nfiFFs/ASbzAKUYSisiunxeOcQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1710335859; a=rsa-sha256; cv=none; b=KTgmJ6+IHvQO78s9BqeaMQxXQvrr7IMmDD6g3J2hrFQpStl21oCuUTNfG0DQZRzNCOmnRe GjrpqRGhMHkLs/4YLO6UAQ51aiI9knpom1vvZ8imdj3VByYuVnJ0w2GmrvH0eQx6G60kbW j4xECRNUI3eY0O7GerUXWXs04NhZsdh9trVmOsJmrvx0XsI1o/CXf7tMSzC5e+trRFvJQq O3YAL48v+pUWuYIta4MvM4FwpnW/2eTl0bxy2bflb1ufCyL6OQVDoabBbVfjZjT3q4Iap3 rS71gICz9Mw9gsvyjDohJyFI8Z9SXiiPjmbDO7LHEZRq1oJ72f8EMQkua4LnQQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1710335859; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=vkt9PPHLo9GRjskLYlv+otGkV2CAA6QQKHyE8/QGnSI=; b=xeqHWRUD2/0Wp7PegbAw7ulA6mEDaolaGsVXGicB2jqyIqWdxksBnoUtkGTlFcB8fIecLu RVprL/OEQavU7gqijX/WHECDsAYPef7G9YVlcjtMmWaQVf+f1/DTPQllCEG5iBYcb+k4kV UaXXJqe17d5A9W/L3Uxy7DZZ/wu2CC9NoshibA7Oyubok4pUpTJYVm9/5U9QdF3R6fywpI O3HxnM3sbvod+RM3Av+axi0uucCop5YOO1/EGf+ko30K8DhfoyWHthKORRNLT1O73XAaLM 0hsCN5d13qKCHYsuWdY8GDOep4GPFaQdRxecVcrZQoLcvZ3E+Bl7yQakdna5/Q== Received: from smtpclient.apple (ns1.oxydns.net [45.32.91.63]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: zlei/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4Tvrf96Szyz12lf; Wed, 13 Mar 2024 13:17:37 +0000 (UTC) (envelope-from zlei@FreeBSD.org) From: Zhenlei Huang Message-Id: Content-Type: multipart/alternative; boundary="Apple-Mail=_25F69FF8-482D-44F0-9161-0D018DB2AD75" List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.8\)) Subject: Re: ipv4 route with ipv6 local link nexthop ? Date: Wed, 13 Mar 2024 21:17:32 +0800 In-Reply-To: <764E12BF-5D31-4905-98AE-6D745BFD1DC2@FreeBSD.org> Cc: "freebsd-net@FreeBSD.org" , Alexander Chernikov To: Benoit Chesneau References: <367504DC-48DA-4DFD-9DB6-CC571F0D26B8@FreeBSD.org> <764E12BF-5D31-4905-98AE-6D745BFD1DC2@FreeBSD.org> X-Mailer: Apple Mail (2.3696.120.41.1.8) --Apple-Mail=_25F69FF8-482D-44F0-9161-0D018DB2AD75 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Mar 13, 2024, at 9:09 PM, Zhenlei Huang wrote: >=20 >=20 >=20 >> On Mar 13, 2024, at 12:19 PM, Zhenlei Huang > wrote: >>=20 >>=20 >>=20 >>> On Mar 13, 2024, at 4:36 AM, Benoit Chesneau = > wrote: >>>=20 >>> On latest freebsd 14.0 release , I am trying to join an ipv4 address = over an IPV6 local link using the following command : >>>=20 >>> `route add -net 10.200.1.1/32 -inet6 = fe80::9439:36ff:fef0:7cbd%vlan200` >>>=20 >>> But I get the following error: >>>=20 >>> ``` >>> route add -net 10.200.1.1/32 -inet6 = fe80::9439:36ff:fef0:7cbd%vlan200 >>> add net 10.200.1.1: gateway fe80::9439:36ff:fef0:7cbd%vlan200 fib 0: = Invalid argument >>=20 >> I guess that is regression by transition of route(8) to NETLINK. >>=20 >> Can you please build sbin/route from source without NETLINK and give = it another try ? >>=20 >> ``` >> # cd /usr/src/sbin/route >> # setenv MK_NETLINK_SUPPORT no >> # make >> ``` >=20 > Hi Benoit, >=20 > I managed to repeat this bug. >=20 > As a workaround, add an IPv4 address to the outgoing interface = (vlan200 in your case), > ``` > # ifconfig vlan200 inet x.x.x.x/y > ``` >=20 > then retry adding the route > ``` > # route add -net 10.200.1.1/32 -inet6 = fe80::9439:36ff:fef0:7cbd%vlan200 > ``` >=20 Hi Alexander, I think the logic of adding route from NETLINK routine is not complete. After bumping netlink debug level, ``` # sysctl net.netlink.debug.nl_route_debug_level=3D9 ``` I got these from kernel message: ``` # dmesg | tail ... [nl_route] finalize_nhop: Unable to determine ifa, skipping [nl_route] rtnl_handle_newroute: Error creating nexthop ... ``` =46rom = https://cgit.freebsd.org/src/tree/sys/netlink/route/rt.c?h=3Dreleng/14.0#n= 753 : ``` /* Both nh_ifp and gateway are set */ if (nh->nh_ifa =3D=3D NULL) { const struct sockaddr *gw_sa =3D &nh->gw_sa; if (gw_sa->sa_family !=3D dst->sa_family) { /* * Use dst as the target for determining the = default * preferred ifa IF * 1) the gateway is link-level (e.g. direct = route) * 2) the gateway family is different (e.g. IPv4 = over IPv6). */ gw_sa =3D dst; } struct ifaddr *ifa =3D ifaof_ifpforaddr(gw_sa, = nh->nh_ifp); if (ifa =3D=3D NULL) { NL_LOG(LOG_DEBUG, "Unable to determine ifa, = skipping"); *perror =3D EINVAL; return (NULL); } nhop_set_src(nh, ifa); } ``` If the outgoing interface `nh->nh_ifp` does not have any IPv4 addresses, = we should try ifaddrs on other interfaces, and have 0.0.0.0 as the last sort ( in case the net stack is IPv6 only, = i.e., no IPv4 addresses available ) >>=20 >>> ``` >>>=20 >>> ping6 to `fe80::9439:36ff:fef0:7cbd%vlan200` works. What I am doing = wrong? Configuration of the interface is the following: >>>=20 >>> ``` >>> vlan200: = flags=3D1008843 metric = 0 mtu 1500 >>> = options=3D4600703 >>> ether fe:d9:dd:2d:35:c5 >>> inet6 fe80::fcd9:ddff:fe2d:35c5%vlan200 prefixlen 64 scopeid = 0x5 >>> groups: vlan >>> vlan: 200 vlanproto: 802.1q vlanpcp: 0 parent interface: = iavf0 >>> media: Ethernet autoselect (10Gbase-SR ) >>> status: active >>> nd6 options=3D23>> ``` >>>=20 >>>=20 >>>=20 >>> Beno=C3=AEt=20 >>=20 >> Best regards, >> Zhenlei --Apple-Mail=_25F69FF8-482D-44F0-9161-0D018DB2AD75 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8

On Mar 13, 2024, at 9:09 PM, Zhenlei Huang <zlei@FreeBSD.org> = wrote:



On= Mar 13, 2024, at 12:19 PM, Zhenlei Huang <zlei@FreeBSD.org> = wrote:



On Mar 13, 2024, at 4:36 AM, = Benoit Chesneau <benoitc@enki-multimedia.eu> wrote:

On latest = freebsd 14.0 release , I am trying to join an ipv4 address over an IPV6 = local link using the following command :

`route add -net = 10.200.1.1/32 -inet6 fe80::9439:36ff:fef0:7cbd%vlan200`

But I  get the = following error:

```
route add -net 10.200.1.1/32 -inet6 = fe80::9439:36ff:fef0:7cbd%vlan200
add net 10.200.1.1: gateway fe80::9439:36ff:fef0:7cbd%vlan200 = fib 0: Invalid argument

I guess that is = regression by transition of route(8) to = NETLINK.

Can you please build = sbin/route from source without NETLINK and give it another try = ?

```
# cd /usr/src/sbin/route
# = setenv MK_NETLINK_SUPPORT no
# make
```

Hi = Benoit,

I managed to repeat this = bug.

As a workaround, add = an IPv4 address to the outgoing interface (vlan200 in your = case),
```
# ifconfig vlan200 = inet x.x.x.x/y
```

then retry adding the route
```
# route add -net 10.200.1.1/32 = -inet6 fe80::9439:36ff:fef0:7cbd%vlan200
```


Hi Alexander,

I = think the logic of adding route from NETLINK routine is not = complete.

After = bumping netlink debug level,
```
# sysctl = net.netlink.debug.nl_route_debug_level=3D9
```

I got these from kernel = message:
```
# dmesg | = tail
...
[nl_route] finalize_nhop: Unable to = determine ifa, skipping
[nl_route] rtnl_handle_newroute: Error = creating nexthop
...
```


```
/* Both nh_ifp and gateway are = set */
= if (nh->nh_ifa =3D=3D NULL) {
= const struct sockaddr *gw_sa =3D &nh->gw_sa;

if (gw_sa->sa_family = !=3D dst->sa_family) {
= /*
* Use dst as the = target for determining the default
= * preferred ifa IF
* 1) the gateway = is link-level (e.g. direct route)
* 2) the gateway = family is different (e.g. IPv4 over IPv6).
= */
gw_sa =3D = dst;
= }

= struct ifaddr *ifa =3D ifaof_ifpforaddr(gw_sa, = nh->nh_ifp);
if (ifa =3D=3D NULL) = {
= NL_LOG(LOG_DEBUG, "Unable to determine ifa, = skipping");
*perror =3D = EINVAL;
return = (NULL);
}
= nhop_set_src(nh, ifa);
}
```
If the = outgoing interface `nh->nh_ifp` does not have any IPv4 = addresses, we should try ifaddrs on other = interfaces,
and have 0.0.0.0 as the last sort ( in case the = net stack is IPv6 only, i.e., no IPv4 addresses available = )



```

ping6 to = `fe80::9439:36ff:fef0:7cbd%vlan200` works. What I am doing = wrong? Configuration of the interface is the following:

```
vlan200: = flags=3D1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> = metric 0 mtu 1500
    =     = options=3D4600703<RXCSUM,TXCSUM,TSO4,TSO6,LRO,RXCSUM_IPV6,TXCSUM_IPV6,M= EXTPG>
    =     ether fe:d9:dd:2d:35:c5
        inet6 = fe80::fcd9:ddff:fe2d:35c5%vlan200 prefixlen 64 scopeid = 0x5
      =   groups: vlan
  =       vlan: 200 vlanproto: 802.1q vlanpcp: 0 parent = interface: iavf0
  =       media: Ethernet autoselect (10Gbase-SR = <full-duplex>)
  =       status: active
  =       nd6 = options=3D23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL
```



Beno=C3=AEt 

Best regards,
Zhenlei
=


= --Apple-Mail=_25F69FF8-482D-44F0-9161-0D018DB2AD75--