Re: vxlan with IPv6 underlay ?

Changing the mtu wasn't enough. To make it works i had to disable vxlan offload features:

```
ifconfig mce0 -vxlanhwcsum -vxlanhwtso
```

Then suddenly all started to work. The card is the following:

```
mlx5_core1@pci0:17:0:1:	class=0x020000 rev=0x00 hdr=0x00 vendor=0x15b3 device=0x1015 subvendor=0x1590 subdevice=0x00d4
    vendor     = 'Mellanox Technologies'
    device     = 'MT27710 Family [ConnectX-4 Lx]'
    class      = network
    subclass   = ethernet
  PCI-e errors = Correctable Error Detected
                 Unsupported Request Detected
     Corrected = Advisory Non-Fatal Error
```

Which is odd. Wonder why it's needed. Thoughts?

Benoît 

On Tuesday, December 5th, 2023 at 13:11, Kristof Provost <kp@FreeBSD.org> wrote:


> On 4 Dec 2023, at 11:25, Benoit Chesneau wrote:
> 
> > Hi,
> > 
> > Is IPv6 underlay fully supported with FreebBSD ? I have created the a tunnel and associated an Ipv6 address to each side. I'm able to ping between each devicesl. But when I want to curl from the remote side it timeout. Locally on the remote side it is OK. Is this expected ? Should I rather create a bridge with vxlan as a member and bind nginx to it ?
> > 
> > ```
> > $ ifconfig vxlan0 create vxlanid 108 vxlanlocal AAAA:BBBB:110b:102::100 vxlanremote AAAA:BBBB:110b:102::12
> > $ ifconfig vxlan0
> > vxlan0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1430
> > options=80020<JUMBO_MTU,LINKSTATE>
> > ether 58:9c:fc:10:ff:eb
> > groups: vxlan
> > vxlan vni 108 local [AAAA:BBBB:102::100]:4789 remote [AAAA:BBBB:110b:102::12]:4789
> > media: Ethernet autoselect (autoselect <full-duplex>)
> > status: active nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
> > $ ifconfig vxlan0 inet6 AAAA:BBBB:110b:300::1/64
> > ````
> > 
> > Ping from remote is ok:
> > `ifconfig vxlan0 vxlan0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1430 options=680323<RXCSUM,TXCSUM,JUMBO_MTU,TSO4,TSO6,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6> ether 58:9c:fc:10:df:1f inet6 fe80::5a9c:fcff:fe10:df1f%vxlan0 prefixlen 64 scopeid 0xf inet6 AAAA:BBBB:110b:300::2 prefixlen 64 groups: vxlan vxlan vni 108 local [AAAA:BBBB110b:102::12]:4789 remote [AAAA:BBBB:110b:102::100]:4789 media: Ethernet autoselect (autoselect <full-duplex>) status: active nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> $ ping6 AAAA:BBBB:110b:300::1 PING6(56=40+8+8 bytes) AAAA:BBBB:110b:300::2 --> AAAA:BBBB::110b:300::1 16 bytes from 2a0e:e701:110b:300::1, icmp_seq=0 hlim=64 time=0.071 ms 16 bytes from 2a0e:e701:110b:300::1, icmp_seq=1 hlim=64 time=0.078 ms 16 bytes from 2a0e:e701:110b:300::1, icmp_seq=2 hlim=64 time=0.076 ms 16 bytes from 2a0e:e701:110b:300::1, icmp_seq=3 hlim=64 time=0.104 ms 16 bytes from 2a0e:e701:110b:300::1, icmp_seq=4 hlim=64 time=0.077 ms^C`
> > 
> > But when I run `curl -6 -v 'http://[AAAA:BBBB:110b:300::1]'` it timeout.
> 
> That smells like an MTU issue. Make sure you’re not blocking ICMPv6, and check with larger ping messages.
> 
> Best regards,
> Kristof