[Bug 268486] panic: vtd_add_device: device 0 is not in scope for any DMA remapping unit

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=268486

            Bug ID: 268486
           Summary: panic: vtd_add_device: device 0 is not in scope for
                    any DMA remapping unit
           Product: Base System
           Version: 13.1-RELEASE
          Hardware: amd64
                OS: Any
            Status: New
          Severity: Affects Some People
          Priority: ---
         Component: kern
          Assignee: bugs@FreeBSD.org
          Reporter: pkubaj@FreeBSD.org

I'm on FreeBSD 13.1-RELEASE-p5 on amd64. My kernel configuration is:
include GENERIC
options DDB                 # required to enable dumps
options KDB_UNATTENDED      # required to enable dumps
options INVARIANTS
options INVARIANT_SUPPORT

nodevice em
nodevice ixl
nodevice iavf
nodevice ice
nodevice ix

I have two Dell R750, one with Intel X722 adapter and another with Intel X710
(both use ixl driver on FreeBSD). After unloading the driver, I'm trying to
passthrough them to the VM.
/boot/loader.conf:
vmm_load="YES"
pptdevs="179/0/0"

pciconf -lv:
ppt0@pci0:179:0:0:      class=0x020000 rev=0x04 hdr=0x00 vendor=0x8086
device=0x37d0 subvendor=0x8086 subdevice=0x0002
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Connection X722 for 10GbE SFP+'
    class      = network
    subclass   = ethernet

I can start VM just fine (with FreeBSD 12.4-RELEASE), however, when I try
starting with the passthrough, I get the following panic on the host:
panic: vtd_add_device: device 0 is not in scope for any DMA remapping unit

Backtrace is:
__curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:55
55              __asm("movq %%gs:%P1,%0" : "=r" (td) : "n" (offsetof(struct
pcpu,
(kgdb) #0  __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:55
#1  doadump (textdump=textdump@entry=0)
    at /usr/src/sys/kern/kern_shutdown.c:399
#2  0xffffffff804b131a in db_dump (dummy=<optimized out>,
    dummy2=<unavailable>, dummy3=<unavailable>, dummy4=<unavailable>)
    at /usr/src/sys/ddb/db_command.c:575
#3  0xffffffff804b11d2 in db_command (last_cmdp=<optimized out>,
    cmd_table=<optimized out>, dopager=dopager@entry=1)
    at /usr/src/sys/ddb/db_command.c:482
#4  0xffffffff804b0e2d in db_command_loop ()
    at /usr/src/sys/ddb/db_command.c:535
#5  0xffffffff804b42a6 in db_trap (type=<optimized out>, code=<optimized out>)
    at /usr/src/sys/ddb/db_main.c:270
#6  0xffffffff80c20c56 in kdb_trap (type=type@entry=3, code=code@entry=0,
    tf=tf@entry=0xfffffe019a333820) at /usr/src/sys/kern/subr_kdb.c:733
#7  0xffffffff810997b9 in trap (frame=0xfffffe019a333820)
    at /usr/src/sys/amd64/amd64/trap.c:607
#8  <signal handler called>
#9  kdb_enter (why=0xffffffff8122185d "panic", msg=<optimized out>)
    at /usr/src/sys/kern/subr_kdb.c:506
#10 0xffffffff80bd37a0 in vpanic (
    fmt=0xffffffff8215136a "vtd_add_device: device %x is not in scope for any
DMA remapping unit", ap=ap@entry=0xfffffe019a333980)
    at /usr/src/sys/kern/kern_shutdown.c:908
#11 0xffffffff80bd3533 in panic (
    fmt=0xffffffff8190aa70 <lock_class_mtx_spin>
"\310\320\022\201\377\377\377\377\n") at /usr/src/sys/kern/kern_shutdown.c:844
#12 0xffffffff82147bc9 in vtd_add_device (arg=<optimized out>, rid=0)
    at /usr/src/sys/amd64/vmm/intel/vtd.c:461
#13 0xffffffff821350fe in IOMMU_ADD_DEVICE (domain=<optimized out>, rid=128)
    at /usr/src/sys/amd64/vmm/io/iommu.c:125
#14 iommu_add_device (dom=<optimized out>, rid=128)
    at /usr/src/sys/amd64/vmm/io/iommu.c:327
#15 iommu_init () at /usr/src/sys/amd64/vmm/io/iommu.c:238
#16 iommu_create_domain (maxaddr=268435456)
    at /usr/src/sys/amd64/vmm/io/iommu.c:271
#17 0xffffffff8212a927 in vm_assign_pptdev (vm=0xfffffe006e7eb000, bus=177,
    slot=0, func=0) at /usr/src/sys/amd64/vmm/vmm.c:988
#18 0xffffffff8212f1aa in vmmdev_ioctl (cdev=<optimized out>, cmd=2148300328,
    data=0xfffffe019a333d50 "\261", fflag=<optimized out>, td=<optimized out>)
    at /usr/src/sys/amd64/vmm/vmm_dev.c:545
#19 0xffffffff80a6a8cc in devfs_ioctl (ap=0xfffffe019a333ba8)
    at /usr/src/sys/fs/devfs/devfs_vnops.c:944
#20 0xffffffff80cd0041 in vn_ioctl (fp=0xfffff8000b47e2d0,
    com=<optimized out>, data=0xfffffe019a333d50,
    active_cred=0xfffff8000bbf6d00, td=0x0)
    at /usr/src/sys/kern/vfs_vnops.c:1696
#21 0xffffffff80a6afae in devfs_ioctl_f (
    fp=0xffffffff8190aa70 <lock_class_mtx_spin>, com=128,
    data=0xffffffff811ed308, cred=0x1, td=0xfffffe01652bf720)
    at /usr/src/sys/fs/devfs/devfs_vnops.c:875
#22 0xffffffff80c44242 in fo_ioctl (fp=<optimized out>, com=2148300328,
    data=0x20, active_cred=0x1, td=0xfffffe01652bf720)
    at /usr/src/sys/sys/file.h:361
#23 kern_ioctl (td=<optimized out>, td@entry=0xfffffe01652bf720,
    fd=<optimized out>, com=com@entry=2148300328,
    data=0x20 <error: Cannot access memory at address 0x20>,
    data@entry=0xfffffe019a333d50 "\261")
    at /usr/src/sys/kern/sys_generic.c:803
#24 0xffffffff80c43f96 in sys_ioctl (td=0xfffffe01652bf720,
    uap=0xfffffe01652bfb08) at /usr/src/sys/kern/sys_generic.c:711
#25 0xffffffff8109a53e in syscallenter (td=<optimized out>)
    at /usr/src/sys/amd64/amd64/../../kern/subr_syscall.c:189
#26 amd64_syscall (td=0xfffffe01652bf720, traced=0)
    at /usr/src/sys/amd64/amd64/trap.c:1185
#27 <signal handler called>
#28 0x0000000801619c2a in ?? ()

-- 
You are receiving this mail because:
You are the assignee for the bug.