mlx5en issue with SR-IOV when using in the bridge

From: Benoit Chesneau <benoitc_at_enki-multimedia.eu>
Date: Sat, 10 Feb 2024 09:23:36 UTC
I have tested the following scenario:

Setup the mellanox driver with SRIOV:

```
$ sudo more /etc/iov/mce0.conf
PF {
device: "mlx5_core0";
num_vfs: 8;
}

DEFAULT {
passthrough: true;
}

VF-0 {
mac-addr: "02:01:02:02:01:00";
}

VF-1 {
mac-addr: "02:01:02:02:01:01";
}

VF-2 {
passthrough: false;}

```

With one VF in the vm answering to a specific vlan, and one jail on the host using vnet with the PF in a bridge and a epair interface added to this bridge.

When I am pinging from the jail to the VF in the vm , the vm receive the ping or arp requests. But the jail doesn't receive the response. In a same manner , when I ping from the VM to the jail, the host is not found. bridge arp doesn't return the vm mac address.

All the hosts has the PF interface set in promiscuous mode but this doesn't seem to be enough to allows the bridge to learn other mac ids. When I directly add the IP to PF, Ping works from the vm to the PF, the reverse is also true.

Do I miss a setting? Is this expected with mellanox cards? What shoudl I look for?

BenoƮt