[Bug 263568] sr-iov: ixv0 connection lost after loading vm with a passthrough device

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=263568

            Bug ID: 263568
           Summary: sr-iov: ixv0 connection lost  after loading vm with a
                    passthrough device
           Product: Base System
           Version: 13.0-RELEASE
          Hardware: Any
                OS: Any
            Status: New
          Severity: Affects Only Me
          Priority: ---
         Component: misc
          Assignee: bugs@FreeBSD.org
          Reporter: benoitc@enki-multimedia.eu

I am trying to use SR-IOV on my machine with an IX driver, but each time i
start the vm on the passthrough devices the connections which is started on the
virtuall function dedicated for the host, the connection hangs and it is not
possible to setup the connection back until I reboot. Do anyone know what i'm
missing in my configuration? I don't see any useful logs in dmesg.

The card is the HP Ethernet 10Gb 2-port 560SFP+ Adapter 665249-B21 which is an
Intel  X520-DAA2.

Maybe related but when restarting  netif I am getting this message: "Mailbox
API negotiation failed in if_init" .I am using the intel-ix-kmod port. I try to
update it to the latest 3.3.30 version which provided the same results.



I've the following configuration to setup sr-iov:

```
more /etc/iov-ix0.conf
PF {
        device: "ix0";
        num_vfs: 4;
}

DEFAULT {
        passthrough: true;
        allow-set-mac: true
        allow-promisc: true;
}

VF-0 {
        passthrough: false;
        mac-addr: "36:6b:d2:f9:a9:08";
}
```

and in loader.conf:

```
ifconfig_ixv0="up"

cloned_interfaces="lo1 bgp services"

vlans_ixv0="bgp services"

create_args_bgp="vlan 1000"
create_args_services="vlan 2000"

ifconfig_bgp_ipv6="inet6 XXX/64"

```

vms starts and the device seeps to be setup:

```
ixv0@pci0:8:0:128:    class=0x020000 rev=0x01 hdr=0x00 vendor=0x8086
device=0x10ed subvendor=0x103c subdevice=0x17d3
    vendor     = 'Intel Corporation'
    device     = '82599 Ethernet Controller Virtual Function'
    class      = network
    subclass   = ethernet
    cap 11[70] = MSI-X supports 3 messages, enabled
                 Table in map 0x1c[0x0], PBA in map 0x1c[0x2000]
    cap 10[a0] = PCI-Express 0 endpoint max data 128(128)
                 max read 128
    ecap 0001[100] = AER 1 0 fatal 0 non-fatal 0 corrected
    ecap 000e[150] = ARI 1
ppt0@pci0:8:0:130:    class=0x020000 rev=0x01 hdr=0x00 vendor=0x8086
device=0x10ed subvendor=0x103c subdevice=0x17d3
    vendor     = 'Intel Corporation'
    device     = '82599 Ethernet Controller Virtual Function'
    class      = network
    subclass   = ethernet
    cap 11[70] = MSI-X supports 3 messages
                 Table in map 0x1c[0x0], PBA in map 0x1c[0x2000]
    cap 10[a0] = PCI-Express 0 endpoint max data 128(128)
                 max read 128
    ecap 0001[100] = AER 1 0 fatal 0 non-fatal 0 corrected
    ecap 000e[150] = ARI 1
ppt1@pci0:8:0:132:    class=0x020000 rev=0x01 hdr=0x00 vendor=0x8086
device=0x10ed subvendor=0x103c subdevice=0x17d3
    vendor     = 'Intel Corporation'
    device     = '82599 Ethernet Controller Virtual Function'
    class      = network
    subclass   = ethernet
    cap 11[70] = MSI-X supports 3 messages
                 Table in map 0x1c[0x0], PBA in map 0x1c[0x2000]
    cap 10[a0] = PCI-Express 0 endpoint max data 128(128)
                 max read 128
    ecap 0001[100] = AER 1 0 fatal 0 non-fatal 0 corrected
    ecap 000e[150] = ARI 1
ppt2@pci0:8:0:134:    class=0x020000 rev=0x01 hdr=0x00 vendor=0x8086
device=0x10ed subvendor=0x103c subdevice=0x17d3
    vendor     = 'Intel Corporation'
    device     = '82599 Ethernet Controller Virtual Function'
    class      = network
    subclass   = ethernet
    cap 11[70] = MSI-X supports 3 messages
                 Table in map 0x1c[0x0], PBA in map 0x1c[0x2000]
    cap 10[a0] = PCI-Express 0 endpoint max data 128(128)
                 max read 128
    ecap 0001[100] = AER 1 0 fatal 0 non-fatal 0 corrected
    ecap 000e[150] = ARI 1
```


VM is started using vm-bhyve:

```
loader="uefi-csm"
cpu=1
memory=2G
passthru0=8/0/130=5:0
disk0_type="virtio-blk"
disk0_name="disk0.img"
uuid="ac3dafab-bedb-11ec-b24d-1402ec690a80"
```

-- 
You are receiving this mail because:
You are the assignee for the bug.