Failed to emulate instruction sequence [ 8b5810f7c340004078740a498b8548 ] at 0xfffff80516c4304a

From: Mario Marietto <marietto2008_at_gmail.com>
Date: Sun, 04 Dec 2022 21:18:06 UTC
Hello.

I would like to inform you about a bug that afflicts bhyve that doesn't let
me run more than one virtual machine at the same time. Let's take this
example to understand when it occurs. I launch the first VM using this
script :

#!/bin/sh

#ps ax | grep bhyve | awk '{ print $1; }' | xargs kill -9
setxkbmap it
vms="$(ls /dev/vmm/*)"
vncs="$(ps ax | awk '/vncviewer [0]/{print $6}')"
for vm in $vms; do
                session="${vm##*/}"
                echo "bhyve session = $session"
                echo "vnc session = $vncs"
                if ! printf '%s\n' "${vncs}" | grep "${session}"; then
                                printf 'VNC session not found,destroying
ghost vms\n'
                                bhyvectl --vm=$session --destroy

                else
                                printf 'Found VNC session %s\n'
"${session},no ghost vms found,not destroying them"
                fi
done

vmdisk=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident: (NM13N4CZ)/
&& d{print d}'`
echo $vmdisk

vmdisk1=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident:
(2015020204055E)/ && d{print d}'`
echo $vmdisk1

vmdisk2=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident:
(2022285F1175)/ && d{print d}'`
echo $vmdisk2

vmdisk3=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident:
(BE0191500218)/ && d{print d}'`
echo $vmdisk3

vmdisk4=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident:
(38434B4237354B45)/ && d{print d}'`
echo $vmdisk4

vmdisk5=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident:
(WD-WCAV2X597309)/ && d{print d}'`
echo $vmdisk5

mount -t ufs /dev/$vmdisk1'p2' /mnt/$vmdisk1'p2'

bhyve -S -c sockets=2,cores=2,threads=2 -m 4G -w -H -A \
-s 0,hostbridge \
-s 1,ahci-hd,/mnt/$vmdisk1'p2'/bhyve/img/Windows/windows11.img,bootindex=1 \
-s 2,ahci-hd,/dev/$vmdisk3 \
-s 3,ahci-hd,/dev/$vmdisk4 \
-s 4,ahci-hd,/dev/$vmdisk5 \
-s 7,virtio-net,tap18 \
-s 21,hda,play=/dev/dsp,rec=/dev/dsp \
-s 29,fbuf,tcp=0.0.0.0:5918,w=1500,h=950 \
-s 30,xhci,tablet \
-s 31,lpc \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_BHF_CODE.fd \
vm18 < /dev/null & sleep 2 && vncviewer 0:18

Windows 11 runs there and it works great. At this point,let's say that I
want to run the VM n. 2 without closing the first one and this time I
choose a Linux vm,that I launch using this script :

#!/bin/sh

setxkbmap it
vms="$(ls /dev/vmm/*)"
vncs="$(ps ax | awk '/vncviewer [0]/{print $6}')"
for vm in $vms; do
                session="${vm##*/}"
                echo "bhyve session = $session"
                echo "vnc session = $vncs"
                if ! printf '%s\n' "${vncs}" | grep "${session}"; then
                                printf 'VNC session not found,destroying
ghost vms\n'
                                bhyvectl --vm=$session --destroy

                else
                                printf 'Found VNC session %s\n'
"${session},no ghost vms found,not destroying them"
                fi
done

#bhyvectl --vm=vm7 --force-reset
#bhyvectl --vm=vm7 --destroy

vmdisk=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident: (NM13N4CZ)/
&& d{print d}'`
echo "Seagate M3 Portable 1.8 TB ; $vmdisk"

vmdisk0=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident:
(2022285F1175)/ && d{print d}'`
echo "CT1000P1SSD8 ; $vmdisk0"

vmdisk1=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident:
(2015020204055E)/ && d{print d}'`
echo "TOSHIBA External USB 3.0 1.8 TB ; $vmdisk1"

vmdisk2=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident:
(20130506005976F)/ && d{print d}'`
echo "TOSHIBA External USB 3.0 932G ; $vmdisk2"

vmdisk3=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident:
(BE0191500218)/ && d{print d}'`
echo "G-DRIVE USB ; $vmdisk3"

vmdisk4=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident:
(38434B4237354B45)/ && d{print d}'`
echo "Elements 25A3 ; $vmdisk4"

vmdisk5=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident:
(WD-WCAV2X597309)/ && d{print d}'`
echo "WDC WD3200AAJS-00L7A0-298 GB ; $vmdisk5"

vmdisk6=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident:
(0774911DDC4200A6)/ && d{print d}'`
echo "SanDisk Cruzer-15GB ; $vmdisk6"

vmdisk7=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident:
(60A44C4138D8F311190A0149)/ && d{print d}'`
echo "Kingston DataTraveler 2.0 ; $vmdisk7"

vmdisk8=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident:
(20140100006C)/ && d{print d}'`
echo "Corsair Force 3 SSD ; $vmdisk8"

vmdisk9=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident:
(1924E20B2AE5)/ && d{print d}'`
echo "CT500MX500SSD4 ; $vmdisk9"

vmdisk10=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident:
(S3Z2NB0KB99028V)/ && d{print d}'`
echo "Samsung SSD 860 EVO ; $vmdisk10"

mount -t ufs /dev/$vmdisk3'p2' /mnt/$vmdisk3'p2'
mount -t ufs /dev/$vmdisk1'p2' /mnt/$vmdisk1'p2'

bhyve -S -c sockets=1,cores=2,threads=2 -m 4G -w -H -A \
-s 0,hostbridge \
-s
2,virtio-blk,/mnt/$vmdisk1'p2'/bhyve/img/Linux/ubuntu2210.img,bootindex=1 \
-s 3,virtio-blk,/dev/$vmdisk4 \
-s 4,virtio-blk,/dev/$vmdisk2 \
-s 5,virtio-blk,/dev/$vmdisk10 \
-s 7,fbuf,tcp=0.0.0.0:5919,w=1600,h=950,wait \
-s 8:0,passthru,2/0/0 \
-s 8:1,passthru,2/0/1 \
-s 8:2,passthru,2/0/2 \
-s 8:3,passthru,2/0/3 \
-s 10,virtio-net,tap19 \
-s 11,virtio-9p,sharename=/ \
-s 30,xhci,tablet \
-s 31,lpc \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI_CODE.fd \
vm0:19 < /dev/null & sleep 2 && vncviewer 0:19

and boom. The first VM crash itself and close giving this error :

Failed to emulate instruction sequence [ 8b5810f7c340004078740a498b8548 ]
at 0xfffff80516c4304a

This is clearly a bug that should be fixed. Maybe @Corvin Köhne
<c.koehne@beckhoff.com> can help here.

Below you can see what are the hardware components of my system :

# lspci

00:00.0 Host bridge: Intel Corporation 8th/9th Gen Core 8-core Desktop
Processor Host Bridge/DRAM Re
gisters [Coffee Lake S] (rev 0d)
00:01.0 PCI bridge: Intel Corporation 6th-10th Gen Core Processor PCIe
Controller (x16) (rev 0d)
00:01.1 PCI bridge: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen
Core Processor PCIe Control
ler (x8) (rev 0d)
00:02.0 Display controller: Intel Corporation CoffeeLake-S GT2 [UHD
Graphics 630] (rev 02)
00:12.0 Signal processing controller: Intel Corporation Cannon Lake PCH
Thermal Controller (rev 10)
00:14.0 USB controller: Intel Corporation Cannon Lake PCH USB 3.1 xHCI Host
Controller (rev 10)
00:14.2 RAM memory: Intel Corporation Cannon Lake PCH Shared SRAM (rev 10)
00:16.0 Communication controller: Intel Corporation Cannon Lake PCH HECI
Controller (rev 10)
00:17.0 SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI
Controller (rev 10)
00:1b.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port
#17 (rev f0)
00:1c.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port
#1 (rev f0)
00:1d.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port
#9 (rev f0)
00:1f.0 ISA bridge: Intel Corporation Z390 Chipset LPC/eSPI Controller (rev
10)
00:1f.3 Audio device: Intel Corporation Cannon Lake PCH cAVS (rev 10)
00:1f.4 SMBus: Intel Corporation Cannon Lake PCH SMBus Controller (rev 10)
00:1f.5 Serial bus controller: Intel Corporation Cannon Lake PCH SPI
Controller (rev 10)
00:1f.6 Ethernet controller: Intel Corporation Ethernet Connection (7)
I219-V (rev 10)
01:00.0 VGA compatible controller: NVIDIA Corporation GP106 [GeForce GTX
1060 3GB] (rev a1)
01:00.1 Audio device: NVIDIA Corporation GP106 High Definition Audio
Controller (rev a1)
02:00.0 VGA compatible controller: NVIDIA Corporation TU102 [GeForce RTX
2080 Ti] (rev a1)
02:00.1 Audio device: NVIDIA Corporation TU102 High Definition Audio
Controller (rev a1)
02:00.2 USB controller: NVIDIA Corporation TU102 USB 3.1 Host Controller
(rev a1)
02:00.3 Serial bus controller: NVIDIA Corporation TU102 USB Type-C UCSI
Controller (rev a1)
03:00.0 Non-Volatile memory controller: Micron/Crucial Technology Device
5403 (rev 03)

-- 
Mario.