From nobody Tue Dec 05 14:21:21 2023 X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Sl2lp6pVNz533cn for ; Tue, 5 Dec 2023 14:21:42 +0000 (UTC) (envelope-from benoitc@enki-multimedia.eu) Received: from mail-40136.proton.ch (mail-40136.proton.ch [185.70.40.136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "protonmail.com", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Sl2lp4gKbz3BxJ; Tue, 5 Dec 2023 14:21:42 +0000 (UTC) (envelope-from benoitc@enki-multimedia.eu) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=enki-multimedia.eu; s=protonmail; t=1701786099; x=1702045299; bh=U9UIer5uauLEQ+kWqBk5CJ5LHUOUD/47CwKTTIBX4H4=; h=Date:To:From:Cc:Subject:Message-ID:In-Reply-To:References: Feedback-ID:From:To:Cc:Date:Subject:Reply-To:Feedback-ID: Message-ID:BIMI-Selector; b=Prq5tH1azWc1Xm4AdLacLc8DoBKoyn7J2Sojn6d0DAKrA0a4P87SqaoVQ78izHhGM TToYUsw6u2Z2rc1za14IBAwZ2pwe5MnXKYJicZJnVMCLr3lYLkb2NfT3LSMD/oBLFx dFzby0C8VA+tts0cJp0JnyAXQzospQgRnkHYBNOffekflnvf4smhjPnsOHIpWIe4eZ USf5c/9HwcQ1KY+pNjuvAI/wklZ/TvyyojLaV+65lSalO4tmnYea5E0rWJsekqD2eA YTCHlWG7piABOEUOYSQ6Fmv5WflnOKOITxUdm8HXmVGyuwpTx8QGhENwfbfhBwwG0r Uc+x+WD3e/SiQ== Date: Tue, 05 Dec 2023 14:21:21 +0000 To: Kristof Provost From: Benoit Chesneau Cc: freebsd-net@freebsd.org Subject: Re: vxlan with IPv6 underlay ? Message-ID: In-Reply-To: <793EA8DF-BD26-414A-A12D-866356D339DB@FreeBSD.org> References: <793EA8DF-BD26-414A-A12D-866356D339DB@FreeBSD.org> Feedback-ID: 9066678:user:proton List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:62371, ipnet:185.70.40.0/24, country:CH] X-Spamd-Bar: ---- X-Rspamd-Queue-Id: 4Sl2lp4gKbz3BxJ Changing the mtu wasn't enough. To make it works i had to disable vxlan off= load features: ``` ifconfig mce0 -vxlanhwcsum -vxlanhwtso ``` Then suddenly all started to work. The card is the following: ``` mlx5_core1@pci0:17:0:1:=09class=3D0x020000 rev=3D0x00 hdr=3D0x00 vendor=3D0= x15b3 device=3D0x1015 subvendor=3D0x1590 subdevice=3D0x00d4 vendor =3D 'Mellanox Technologies' device =3D 'MT27710 Family [ConnectX-4 Lx]' class =3D network subclass =3D ethernet PCI-e errors =3D Correctable Error Detected Unsupported Request Detected Corrected =3D Advisory Non-Fatal Error ``` Which is odd. Wonder why it's needed. Thoughts? Beno=C3=AEt=20 On Tuesday, December 5th, 2023 at 13:11, Kristof Provost w= rote: > On 4 Dec 2023, at 11:25, Benoit Chesneau wrote: >=20 > > Hi, > >=20 > > Is IPv6 underlay fully supported with FreebBSD ? I have created the a t= unnel and associated an Ipv6 address to each side. I'm able to ping between= each devicesl. But when I want to curl from the remote side it timeout. Lo= cally on the remote side it is OK. Is this expected ? Should I rather creat= e a bridge with vxlan as a member and bind nginx to it ? > >=20 > > ``` > > $ ifconfig vxlan0 create vxlanid 108 vxlanlocal AAAA:BBBB:110b:102::100= vxlanremote AAAA:BBBB:110b:102::12 > > $ ifconfig vxlan0 > > vxlan0: flags=3D1008843 metric 0 mtu 1430 > > options=3D80020 > > ether 58:9c:fc:10:ff:eb > > groups: vxlan > > vxlan vni 108 local [AAAA:BBBB:102::100]:4789 remote [AAAA:BBBB:110b:10= 2::12]:4789 > > media: Ethernet autoselect (autoselect ) > > status: active nd6 options=3D29 > > $ ifconfig vxlan0 inet6 AAAA:BBBB:110b:300::1/64 > > ```` > >=20 > > Ping from remote is ok: > > `ifconfig vxlan0 vxlan0: flags=3D1008843 metric 0 mtu 1430 options=3D680323 ether 58:9c:fc:10:df:1f ine= t6 fe80::5a9c:fcff:fe10:df1f%vxlan0 prefixlen 64 scopeid 0xf inet6 AAAA:BBB= B:110b:300::2 prefixlen 64 groups: vxlan vxlan vni 108 local [AAAA:BBBB110b= :102::12]:4789 remote [AAAA:BBBB:110b:102::100]:4789 media: Ethernet autose= lect (autoselect ) status: active nd6 options=3D21 $ ping6 AAAA:BBBB:110b:300::1 PING6(56=3D40+8+8 bytes) AAAA= :BBBB:110b:300::2 --> AAAA:BBBB::110b:300::1 16 bytes from 2a0e:e701:110b:3= 00::1, icmp_seq=3D0 hlim=3D64 time=3D0.071 ms 16 bytes from 2a0e:e701:110b:= 300::1, icmp_seq=3D1 hlim=3D64 time=3D0.078 ms 16 bytes from 2a0e:e701:110b= :300::1, icmp_seq=3D2 hlim=3D64 time=3D0.076 ms 16 bytes from 2a0e:e701:110= b:300::1, icmp_seq=3D3 hlim=3D64 time=3D0.104 ms 16 bytes from 2a0e:e701:11= 0b:300::1, icmp_seq=3D4 hlim=3D64 time=3D0.077 ms^C` > >=20 > > But when I run `curl -6 -v 'http://[AAAA:BBBB:110b:300::1]'` it timeout= . >=20 > That smells like an MTU issue. Make sure you=E2=80=99re not blocking ICMP= v6, and check with larger ping messages. >=20 > Best regards, > Kristof