Re: Traffic between cxgbe VFs and/or PF on a host

On Fri, Oct 11, 2024 at 3:56 PM John Nielsen <lists@jnielsen.net> wrote:

> Hi-
>
> I’m running a FreeBSD 14-STABLE host with a Chelstio T520. I have a bhyve
> VM (also running 14-STABLE) to which I have assigned a VF of the NIC. That
> is all working as expected; the host can pass traffic using the PF cxl0 and
> the guest can pass traffic using the VF cxlv0. However the host cannot
> communicate with the guest. I am looking in to the possibility of enabling
> 802.1qbg / VEPA / reflective relay on the switch port but I’d like to know
> if the T5 can do that switching itself without sending the packets over the
> wire. The marketing material says the card "integrates a high performance
> packet switch” but I don’t know how to configure that functionality on
> FreeBSD or if this use case is supported. Can anyone shed some light on
> that?
>

The PF driver's tx bypasses the internal switch by default and is not
visible to the VFs because of that.  Set this knob to force it go through
the switch.

     hw.cxgbe.tx_vm_wr
             Setting this to 1 instructs the driver to use VM work requests
to transmit data.
             This lets PF interfaces transmit frames to VF interfaces over
the internal switch in
             the ASIC.  Note that the cxgbev(4) VF driver always uses VM
work requests and is not
             affected by this tunable.  The default value is 0 and should
be changed only if PF
             and VF interfaces need to communicate with each other.
Different interfaces can be
             assigned different values using the dev.<port>.X.tx_vm_wr
sysctl when the interface
             is administratively down.

Regards,
Navdeep


> The other alternative would be to wire up the second port but if I can get
> away with not needing to use another SFP+ port on the switch for this that
> would be ideal.
>
> Thanks!
>
> JN
>
>
>