Re: ERRATA CORRIGE "BUG !!!! X64 Exception Type - 06(#UD - Invalid Opcode) CPU Apic ID - 00000000 !!!! while trying to pass thru my devices with bhyve on FreeBSD 14-CURRENT"
Date: Thu, 10 Feb 2022 14:39:46 UTC
Deleted all the screen dumps since my question is more general: What happens if you do not turn on Xorg on the host and start the VM from the console (likely should make no difference but worth a try). In your original post you never mentioned it worked on 14-CURRENT but not on 13-RELEASE (w/ patches). If this is the case it is almost certainly a bug, but the general policy of most developers is not to make bug fixes for older versions. This means you are likely stuck with 14-CURRENT. On Thu, Feb 10, 2022 at 9:24 AM Mario Marietto <marietto2008@gmail.com> wrote: > Hello. > > I've bought an additional PCI to USB (Renesas) controller that I've > attached to the PCIe Gen3 x4 (Type 22110) of my MOBO Gigabyte Aorus Pro > with Intel I9 chipset and Nvidia Geforce RTX 2080 ti. On this controller > I've attached two USB disks and my KInect 2. The nVidia geforce RTX 2080 ti > is attached to the PCIe Gen3 x 4 (Type 2280). Check this image : > https://ibb.co/fC0jR1p > > check by yourself the logs showing that my passed through devices are > working great : > > dmesg --> https://pastebin.ubuntu.com/p/cSy7jkHdZz/ > pciconf -vl --> https://pastebin.ubuntu.com/p/xZNggrz6D5/ > > On FreeBSD 14 the bhyve setup below works. I can boot Linux passing > through the devices that you see. But I have used a raw image file (not a > physical installation on a sata or nvme disk,because in this case it wont > work and for this I've opened another bug). With this I want to demonstrate > that there aren't any problems with my bios settings and with my passed > thru devices. > > bhyve -S -c sockets=2,cores=2,threads=2 -m 8G -w -H \ > -s 0,hostbridge \ > -s 1,virtio-blk,/mnt/da1p2/bhyve/os/Linux/impish-cuda-11-5-nvidia-495.img \ > -s 3:0,passthru,2/0/0 \ > -s 3:1,passthru,2/0/1 \ > -s 3:2,passthru,2/0/2 \ > -s 3:3,passthru,2/0/3 \ > -s 4:0,passthru,1/0/0 \ > -s 9,virtio-net,tap0 \ > -s 29,fbuf,tcp=0.0.0.0:5900,w=1440,h=900 \ > -s 30,xhci,tablet \ > -s 31,lpc \ > -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \ > -l com1,stdio \ > vm0 > > My host graphic card is not an Nvidia card. It's the graphic chipset > integrated on the motherboard (CoffeeLake-S GT2 [UHD Graphics 630); to use > it I've installed the xf86-video-intel driver with this xorg.conf file : > > Section "Device" > Identifier "Card0" > Driver "intel" > BusID "PCI:0:2:0" > EndSection > > so,now,let's to straight to this problem : > > I've created a new VM with the goal to passthru my graphic card and my usb > controller from the host os (freebsd 14-CURRENT) to the guest os (freebsd > 13R p7) and I've used the parameters below : > > bhyve -S -c sockets=1,cores=2,threads=2 -m 4G -w -H -A \ > -s 0,hostbridge \ > -s 1,ahci-hd,/mnt/da2p2/bhyve/os/BSD/freebsd13vm.img \ > -s 2,passthru,1/0/0 \ > -s 3:0,passthru,2/0/0 \ > -s 3:1,passthru,2/0/1 \ > -s 3:2,passthru,2/0/2 \ > -s 3:3,passthru,2/0/3 \ > -s 8,virtio-net,tap1 \ > -s 9,virtio-9p,sharename=/ \ > -s 29,fbuf,tcp=0.0.0.0:5901,w=1440,h=900,wait \ > -s 30,xhci,tablet \ > -s 31,lpc \ > -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \ > -l com1,stdio \ > vm1 > > inside /etc/rc.conf I've added the following parameters : > > linux_enable="YES" > linux_mounts_enable="YES" > vm_enable="YES" > kdm5_enable="YES" > dbus_enable="YES" > sddm_enable="YES" > > on /boot/loader.conf : > > nvidia_load="YES" > vmm_load="YES" > nmdm_load="YES" > if_tap_load="YES" > if_bridge_load="YES" > bridgestp_load="YES" > linprocfs_load="YES" > linsysfs_load="YES" > tmpfs_load="YES" > verbose_loading="YES" > hw.vmm.topology.cores_per_package=8 > kern.racct.enable=1 > snd_hda_load="YES" > > on /etc/sysctl.conf > > net.link.tap.up_on_open=1 > net.inet.ip.forwarding=1 > net.inet.ip.random_id=1 > kern.evdev.rcpt_mask=6 > kern.init_shutdown_timeout="900" > kern.capmode_coredump=1 > > and with this xorg.conf file : > > Section "ServerLayout" > Identifier "X.org Configured" > Screen 0 "Screen0" 0 0 > #Screen 1 "Screen1" RightOf "Screen0" > InputDevice "Mouse0" "CorePointer" > InputDevice "Keyboard0" "CoreKeyboard" > EndSection > > Section "Module" > Load "glx" > EndSection > > Section "Files" > ModulePath "/usr/local/lib/xorg/modules" > FontPath "/usr/local/share/fonts/misc/" > FontPath "/usr/local/share/fonts/TTF/" > FontPath "/usr/local/share/fonts/OTF/" > FontPath "/usr/local/share/fonts/Type1/" > FontPath "/usr/local/share/fonts/100dpi/" > FontPath "/usr/local/share/fonts/75dpi/" > FontPath "catalogue:/usr/local/etc/X11/fontpath.d" > EndSection > > Section "InputDevice" > Identifier "Mouse0" > Driver "mouse" > Option "Protocol" "auto" > Option "Device" "/dev/sysmouse" > Option "ZAxisMapping" "4 5 6 7" > EndSection > > Section "Monitor" > Identifier "Monitor0" > VendorName "Monitor Vendor" > ModelName "Monitor Model" > EndSection > > Section "Device" > ### Available Driver options are:- > ### Values: <i>: integer, <f>: float, <bool>: "True"/"False", > ### <string>: "String", <freq>: "<f> Hz/kHz/MHz", > ### <percent>: "<f>%" > ### [arg]: arg optional > #Option "SWcursor" # [<bool>] > #Option "kmsdev" # <str> > #Option "ShadowFB" # [<bool>] > #Option "AccelMethod" # <str> > #Option "PageFlip" # [<bool>] > #Option "ZaphodHeads" # <str> > #Option "DoubleShadow" # [<bool>] > #Option "Atomic" # [<bool>] > Identifier "Card0" > Driver "nvidia" > BusID "PCI:0:4:0" > EndSection > > this is what happens when I boot it : > > Loading kernel... > /boot/kernel/kernel text=0x17b9e0 text=0xdd6d50 text=0x65ba0c data=0x140 > data=0x1b9348+0x445cb8 syms=[0x8+0x178ed8+0x8+0x1990a2] > Loading configured modules... > snd_hda...if_bridge.../boot/kernel/if_bridge.ko size 0xfc48 at 0x2112000 > loading required module 'bridgestp' > /boot/kernel/bridgestp.ko size 0x8178 at 0x2122000 > tmpfs.../boot/kernel/tmpfs.ko size 0x14820 at 0x212b000 > /etc/hostid.../etc/hostid size=0x25 > /boot/entropy.../boot/entropy size=0x1000 > if_tap...linprocfs.../boot/kernel/linprocfs.ko size 0x10e50 at 0x2141000 > loading required module 'linux_common' > /boot/kernel/linux_common.ko size 0x2c650 at 0x2152000 > nvidia.../boot/modules/nvidia.ko size 0x28c8640 at 0x217f000 > loading required module 'linux' > /boot/kernel/linux.ko size 0x9e070 at 0x4a48000 > nmdm.../boot/kernel/nmdm.ko size 0x4110 at 0x4ae7000 > bridgestp...linsysfs.../boot/kernel/linsysfs.ko size 0x79b8 at 0x4aec000 > vmm.../boot/kernel/vmm.ko size 0x5748f0 at 0x4af4000 > Start @ 0xffffffff8037c000 ... > efi_check_space: Unable to expand staging area > efi_check_space: Unable to expand staging area > EFI framebuffer information: > addr, size 0xc3000000, 0x1000000 > dimensions 1440 x 900 > stride 1440 > masks 0x00ff0000, 0x0000ff00, 0x000000ff, 0xff000000 > !!!! X64 Exception Type - 06(#UD - Invalid Opcode) CPU Apic ID - 00000000 > !!!! > RIP - 00000000000C000E, CS - 0000000000000038, RFLAGS - 0000000000010086 > RAX - 0000000054A218D8, RCX - 0000000000000002, RDX - 0000000000000002 > RBX - 0000000005069001, RSP - 00000000BFF58098, RBP - 00000000BFF58100 > RSI - 0000000000001000, RDI - 0000000040001000 > R8 - 0000000000000001, R9 - 00000000BFF580D8, R10 - 00000000B902D6E0 > R11 - 0000000000000D80, R12 - 00000000BE819050, R13 - 0000000005069000 > R14 - 00000000BE819058, R15 - 00000000BE819048 > DS - 0000000000000030, ES - 0000000000000030, FS - 0000000000000030 > GS - 0000000000000030, SS - 0000000000000030 > CR0 - 0000000080010033, CR2 - 0000000000000000, CR3 - 00000000BFC01000 > CR4 - 0000000000000668, CR8 - 0000000000000000 > DR0 - 0000000000000000, DR1 - 0000000000000000, DR2 - 0000000000000000 > DR3 - 0000000000000000, DR6 - 00000000FFFF0FF0, DR7 - 0000000000000400 > GDTR - 00000000BFBEEA98 0000000000000047, LDTR - 0000000000000000 > IDTR - 00000000BF832018 0000000000000FFF, TR - 0000000000000000 > FXSAVE_STATE - 00000000BFF57CF0 > !!!! Can't find image information. !!!! > > it freezes here and I can't do anything anymore. > > -- > Mario. > -- Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org