From nobody Wed Jan 17 12:29:59 2024 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TFQFB00Gvz57kD1 for ; Wed, 17 Jan 2024 12:30:06 +0000 (UTC) (envelope-from michael@sloged.al) Received: from exchange.dots.no (vmi1067082.contaboserver.net [109.123.240.255]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4TFQF93SpWz4mJB; Wed, 17 Jan 2024 12:30:05 +0000 (UTC) (envelope-from michael@sloged.al) Authentication-Results: mx1.freebsd.org; none Received: from exchange.dots.no (localhost6.localdomain6 [IPv6:::1]) by exchange.dots.no (Postfix) with ESMTP id AD28F7062806; Wed, 17 Jan 2024 13:30:00 +0100 (CET) List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 From: "=?utf-8?B?TWljaGFlbCBTbMO4Z2VkYWw=?=" To: "=?utf-8?B?Q29ydmluIEvDtmhuZQ==?=" , "freebsd-virtualization@FreeBSD.org" Importance: Normal Sensitivity: Normal Date: Wed, 17 Jan 2024 13:29:59 +0100 Subject: RE: bhyve intel 530 integrated graphics error Thread-Topic: bhyve intel 530 integrated graphics error Thread-Index: ARVJ2gGG6AMMiOd950GC0AkYWiWDj/////9Rif6gHtA= Message-ID: References: In-Reply-To: Content-Language: en-us X-MS-Exchange-Organization-SCL: -1 Content-ID: <920fdbe98365864e8001e85291b8f864@sloged.al> X-Mailer: gromox-emsmdb 2.9.69.fd54100 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0005_B98A84ED.F06AD2C" X-Rspamd-Queue-Id: 4TFQF93SpWz4mJB X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:51167, ipnet:109.123.240.0/20, country:DE] This is a multi-part message in MIME format. ------=_NextPart_000_0005_B98A84ED.F06AD2C Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Thanks Corvin,=0A=0A =0A=0AHowever it=E2=80=99s still not clear to me fro= m D26209 what exactly I need to do, and=0Awhat I=E2=80=99ve tried doesn=E2= =80=99t work =F0=9F=98=8A Any additional advice is appreciated.=0A=0A =0A= =0A 1. Dropped the GOP rom for now. Unless I need it for windows loading = screen /=0A windows diagnostics/repair ? GpuZ won=E2=80=99t extract it= anyhow.=0A 2. I=E2=80=99ve added the -A option, no visible change=0A 3. = BHYVE_CODE.fd =E2=80=93 not sure if I=E2=80=99m using it right. I changed= the symlink for=0A BHYVE_UEFI.fd to point to this file. No visible ch= ange =E2=80=93 except I now have=0A 4 non-working COM ports.=0A 4. I t= ried passthru on usb, and get the same error that I did with the audio=0A= card (see below)=0A 5. The gfx card initially boots with windows Erro= r 43. When I delete it and=0A scan for hardware changes it pops back i= n, without the error, but no=0A displays detected.=0A 6. I=E2=80=99m u= sing vga/dvi ports without success, haven=E2=80=99t tried displayport. No= hdmi=0A ports. In case that matters.=0A 7. Should I try to update fre= ebsd to 14-STABLE branch and see if that makes a=0A difference? (If I = can figure out how)=0A=0A =0A=0A# tail /vm/win10ltsc/bhyve.log=0A=0Apci_f= buf: mmap_memseg failed=0A=0AAssertion failed: (mr->name =3D=3D memp->nam= e), function unregister_mem, file /usr/=0Asrc/usr.sbin/bhyve/mem.c, line = 344.=0A=0A =0A=0A# tail /vm/win10ltsc/vm-bhyve.log=0A=0AJan 17 13:08:48:=C2= =A0 [primary disk: disk0.img]=0A=0AJan 17 13:08:48:=C2=A0 [primary disk d= ev: file]=0A=0AJan 17 13:08:48: dynamically allocated port 5900 for vnc c= onnections=0A=0AJan 17 13:08:48: booting=0A=0AJan 17 13:08:48:=C2=A0 [bhy= ve options: -c 6,sockets=3D1,cores=3D3,threads=3D2 -m 8G -Hwl=0Abootrom,/= usr/local/share/uefi-firmware/BHYVE_UEFI.fd -A -U=0A8dff5c4a-b083-11ee-b2= cd-78d004259091 -S]=0A=0AJan 17 13:08:48:=C2=A0 [bhyve devices: -s 0,host= bridge -s 31,lpc -s 4:0,ahci,hd:/vm/=0Awin10ltsc/disk0.img -s 31:6,passth= ru,0/31/6 -s 2:0,passthru,0/2/0 -s=0A20:0,passthru,0/20/0 -s 6:0,fbuf,tcp= =3D0.0.0.0:5900,w=3D1920,h=3D1080 -s=0A7:0,xhci,tablet]=0A=0AJan 17 13:08= :48:=C2=A0 [bhyve console: -l com1,/dev/nmdm-win10ltsc.1A]=0A=0AJan 17 13= :08:48: starting bhyve (run 1)=0A=0AJan 17 13:08:50: bhyve exited with st= atus 134=0A=0AJan 17 13:08:50: stopped=0A=0A =0A=0A# pciconf -lv |grep -E= C 2 "(USB 3|Audio)"=0A=0Appt1@pci0:0:20:0:=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0 class=3D0x0c0330 rev=3D0x31 hdr=3D0x00 vendor=3D0x8086 device=3D=0A0x= a12f subvendor=3D0x8086 subdevice=3D0xa12f=0A=0A=C2=A0=C2=A0=C2=A0 vendor= =C2=A0=C2=A0=C2=A0=C2=A0 =3D 'Intel Corporation'=0A=0A=C2=A0=C2=A0=C2=A0 = device=C2=A0=C2=A0=C2=A0=C2=A0 =3D '100 Series/C230 Series Chipset Family= USB 3.0 xHCI=0AController'=0A=0A=C2=A0=C2=A0=C2=A0 class=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0 =3D serial bus=0A=0A=C2=A0=C2=A0=C2=A0 subclass=C2=A0=C2=A0 = =3D USB=0A=0A--=0A=0Appt2@pci0:0:31:3:=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= class=3D0x040300 rev=3D0x31 hdr=3D0x00 vendor=3D0x8086 device=3D=0A0xa17= 0 subvendor=3D0x8086 subdevice=3D0xa170=0A=0A=C2=A0=C2=A0=C2=A0 vendor=C2= =A0=C2=A0=C2=A0=C2=A0 =3D 'Intel Corporation'=0A=0A=C2=A0=C2=A0=C2=A0 dev= ice=C2=A0=C2=A0=C2=A0=C2=A0 =3D '100 Series/C230 Series Chipset Family HD= Audio Controller'=0A=0A=C2=A0=C2=A0=C2=A0 class=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0 =3D multimedia=0A=0A=C2=A0=C2=A0=C2=A0 subclass=C2=A0=C2=A0 =3D HDA=0A= =0A =0A=0A =0A=0AFrom: Corvin K=C3=B6hne =0ASent: We= dnesday, January 17, 2024 8:19 AM=0ATo: Michael Sl=C3=B8gedal ; freebsd-virtualization@FreeBSD.org=0ASubject: Re: bhyve intel 5= 30 integrated graphics error=0A=0A =0A=0AHi,=0A=0A =0A=0Aunfortunately, G= PU passthrough is supported by bhyve but not OVMF. So,=0A=0Ayou have to u= se a patched OVMF. The required patch stack is available=0A=0Aat https://= reviews.freebsd.org/D41714. An older version and most=0A=0Aprobably still= working prebuilt version is available at=0A=0Ahttps://reviews.freebsd.or= g/D26209.=0A=0A =0A=0AOn Wed, 2024-01-17 at 01:18 +0100, Michael Sl=C3=B8= gedal wrote:=0A=0A> Hi,=0A=0A> =0A=0A> I=E2=80=99ve been trying to colle= ct the necessary info online about how to do=0A=0A> gpu passthru with int= egrated intel graphics to win10, but can=E2=80=99t make=0A=0A> it work.=0A= =0A> =0A=0A> 1. First tried with just passthru, and after 3-4 reboots= , the=0A=0A> display adapter shows up in win10 guest, but it won=E2=80=99= t load properly.=0A=0A =0A=0ADespite some configuration requirements (e.g= =2E always use slot 2) that's=0A=0Aall you need.=0A=0A =0A=0A> 2. Trie= d adding the hd audio device as well, get bluescreen with=0A=0A> =E2=80=9C= memory_management=E2=80=9D stop code in guest=0A=0A =0A=0AShouldn't happe= n but seems like another issue.=0A=0A =0A=0A> 3. Tried adding a gop ro= m from=0A=0A> https://github.com/patmagauran/i915ovmfPkg - host crashed a= nd got=0A=0A> error below=0A=0A =0A=0AA GOP is only required for pre OS g= raphics (e.g. grub menu).=0A=0A =0A=0AI highly recommend using the GOP fr= om your host system to avoid damages=0A=0Aor strange issues due to incomp= atibilities. Unfortunately, it's not=0A=0Aalways possible to easily extra= ct the GOP from your host system. Even=0A=0AIntel recommends to ask your = board manufacturer:=0A=0Ahttps://projectacrn.github.io/3.2/tutorials/gpu-= passthru.html#enable-the-gvt-d-gop-driver=0A=0A =0A=0ASome alternative wa= ys to extract the GOP:=0A=0A =0A=0AOn Windows try GPU-Z:=0A=0Ahttps://nvi= dia.custhelp.com/app/answers/detail/a_id/4188/~/extracting-the-geforce-vi= deo-bios-rom-file=0A=0AOn Linux try sysfs:=0A=0Ahttps://pve.proxmox.com/w= iki/PCI_Passthrough#The_.27romfile.27_option=0A=0A =0A=0A> =0A=0A> I=E2=80= =99m at a loss what to try next. Any ideas where I=E2=80=99m going wrong?= =0A=0A> =0A=0A> # tail /vm/win10ltsc/vm-bhyve.log =0A=0A> Jan 17 0= 0:49:02: [primary disk: disk0.img]=0A=0A> Jan 17 00:49:02: [primary dis= k dev: file]=0A=0A> Jan 17 00:49:02: dynamically allocated port 5900 for = vnc connections=0A=0A> Jan 17 00:49:02: booting=0A=0A> Jan 17 00:49:02: = [bhyve options: -c 6,sockets=3D1,cores=3D3,threads=3D2 -m=0A=0A> 8G -Hwl = bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd -s=0A=0A> 2:0,passth= ru,0/2/0,rom=3D/vm/win10ltsc/i915ovmf.rom -U 8dff5c4a-b083-=0A=0A> 11ee-b= 2cd-78d004259091 -S]=0A=0A> Jan 17 00:49:02: [bhyve devices: -s 0,hostbr= idge -s 31,lpc -s=0A=0A> 4:0,ahci,hd:/vm/win10ltsc/disk0.img -s 31:6,pass= thru,0/31/6 -s=0A=0A> 6:0,fbuf,tcp=3D0.0.0.0:5900,w=3D1920,h=3D1080 -s 7:= 0,xhci,tablet]=0A=0A> Jan 17 00:49:02: [bhyve console: -l com1,/dev/nmdm= -win10ltsc.1A]=0A=0A =0A=0AYou have to use the `-A` flag. There's an open= PR for vm-bhyve:=0A=0Ahttps://github.com/churchers/vm-bhyve/pull/525/com= mits/653d01f5b53c0bd8ce707137f67617536de8b1c0=0A=0A =0A=0A> Jan 17 00:49:= 02: starting bhyve (run 1)=0A=0A> Jan 17 00:49:04: bhyve exited with stat= us 134=0A=0A> Jan 17 00:49:04: stopped=0A=0A> =0A=0A> # tail /vm/win10lt= sc/bhyve.log =0A=0A> pci_fbuf: mmap_memseg failed=0A=0A> Assertion faile= d: (mr->name =3D=3D memp->name), function unregister_mem,=0A=0A> file /us= r/src/usr.sbin/bhyve/mem.c, line 344.=0A=0A> =0A=0A> # uname -a=0A=0A> F= reeBSD yellowfellow 14.0-RELEASE-p3 FreeBSD 14.0-RELEASE-p3 #0: Mon=0A=0A= > Dec 11 04:56:01 UTC 2023 =0A=0A> root@amd64-builder.daemonology.net:= /usr/obj/usr/src/amd64.amd64/sys/G=0A=0A> ENERIC amd64=0A=0A =0A=0A--=0A=0A= Kind regards,=0A=0ACorvin=0A=0A ------=_NextPart_000_0005_B98A84ED.F06AD2C Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset="utf-8"

Thanks Corvin,

 

However it=E2=80=99s still = not clear to me from D26209 what exactly I need to do, and what I=E2=80=99= ve tried doesn=E2=80=99t work 😊 Any additional advice is appreciated.<= /o:p>

 

  1. Dropped the GOP rom for now. Unless I nee= d it for windows loading screen / windows diagnostics/repair ? GpuZ won=E2= =80=99t extract it anyhow.
  2. I=E2=80=99ve added the -A= option, no visible change
  3. BHYVE_CODE.fd =E2=80=93 n= ot sure if I=E2=80=99m using it right. I changed the symlink for BHYVE_UE= FI.fd to point to this file. No visible change =E2=80=93 except I now hav= e 4 non-working COM ports.
  4. I tried passthru on usb, = and get the same error that I did with the audio card (see below)
  5. The gfx card initially boots with windows Error 43. When I = delete it and scan for hardware changes it pops back in, without the erro= r, but no displays detected.
  6. I=E2=80=99m using vga/d= vi ports without success, haven=E2=80=99t tried displayport. No hdmi port= s. In case that matters.
  7. Should I try to update free= bsd to 14-STABLE branch and see if that makes a difference? (If I can fig= ure out how)

 

# tail /vm/win10ltsc/bhyve.log

pci_fbuf: mmap_memseg failed

Assertion failed: (mr->name =3D=3D memp->name), function un= register_mem, file /usr/src/usr.sbin/bhyve/mem.c, line 344.

 

# tail /v= m/win10ltsc/vm-bhyve.log

Jan 17 13:08:= 48:=C2=A0 [primary disk: disk0.img]

Ja= n 17 13:08:48:=C2=A0 [primary disk dev: file]

Jan 17 13:08:48: dynamically allocated port 5900 for vnc connecti= ons

Jan 17 13:08:48: booting

Jan 17 13:08:48:=C2=A0 [bhyve options: -c 6,soc= kets=3D1,cores=3D3,threads=3D2 -m 8G -Hwl bootrom,/usr/local/share/uefi-f= irmware/BHYVE_UEFI.fd -A -U 8dff5c4a-b083-11ee-b2cd-78d004259091 -S]=

Jan 17 13:08:48:=C2=A0 [bhyve devices: -s = 0,hostbridge -s 31,lpc -s 4:0,ahci,hd:/vm/win10ltsc/disk0.img -s 31:6,pas= sthru,0/31/6 -s 2:0,passthru,0/2/0 -s 20:0,passthru,0/20/0 -s 6:0,fbuf,tc= p=3D0.0.0.0:5900,w=3D1920,h=3D1080 -s 7:0,xhci,tablet]

Jan 17 13:08:48:=C2=A0 [bhyve console: -l com1,/dev/nmdm= -win10ltsc.1A]

Jan 17 13:08:48: starti= ng bhyve (run 1)

Jan 17 13:08:50: bhyv= e exited with status 134

Jan 17 13:08:= 50: stopped

 

# pciconf -lv |grep -EC 2 "(USB 3|Audio)"<= /o:p>

ppt1@pci0:0:20:0:=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 class=3D0x0c0330 rev=3D0x31 hdr=3D0x00 vendor=3D0x8086 device=3D= 0xa12f subvendor=3D0x8086 subdevice=3D0xa12f

=C2=A0=C2=A0=C2=A0 vendor=C2=A0=C2=A0=C2=A0=C2=A0 =3D 'Intel Corpo= ration'

=C2=A0=C2=A0=C2=A0 device=C2=A0= =C2=A0=C2=A0=C2=A0 =3D '100 Series/C230 Series Chipset Family USB 3.0 xHC= I Controller'

=C2=A0=C2=A0=C2=A0 class= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 =3D serial bus

=C2=A0=C2=A0=C2=A0 subclass=C2=A0=C2=A0 =3D USB

--

ppt2@pci0:0:31:3:= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 class=3D0x040300 rev=3D0x31 hdr=3D0x= 00 vendor=3D0x8086 device=3D0xa170 subvendor=3D0x8086 subdevice=3D0xa170<= o:p>

=C2=A0=C2=A0=C2=A0 vendor=C2=A0=C2=A0=C2= =A0=C2=A0 =3D 'Intel Corporation'

=C2=A0= =C2=A0=C2=A0 device=C2=A0=C2=A0=C2=A0=C2=A0 =3D '100 Series/C230 Series C= hipset Family HD Audio Controller'

=C2= =A0=C2=A0=C2=A0 class=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 =3D multimedia

=C2=A0=C2=A0=C2=A0 subclass=C2=A0=C2=A0 =3D = HDA

 

 

From= : Corvin K=C3=B6hne <corvink@FreeBSD.org>
Sent: Wedn= esday, January 17, 2024 8:19 AM
To: Michael Sl=C3=B8gedal <m= ichael@sloged.al>; freebsd-virtualization@FreeBSD.org
Subject: Re: bhyve intel 530 integrated graphics error

 

Hi,
 
unfortunately, GPU passthrough is supported =
by bhyve but not OVMF. So,
you have to use a patched=
 OVMF. The required patch stack is available
at https://reviews.freebsd.org/D4=
1714. An older version and most
probably still w=
orking prebuilt version is available at
https://reviews.freebsd.org/D26209=
=2E
 
On Wed, 2024-01-17 a=
t 01:18 +0100, Michael Sl=C3=B8gedal wrote:
> Hi,=

>  
> I=E2=80=99ve =
been trying to collect the necessary info online about how to do
> gpu passthru with integrated intel graphics to win10, b=
ut can=E2=80=99t make
> it work.
= 
>  
>    1. First tri=
ed with just passthru, and after 3-4 reboots, the
&g=
t; display adapter shows up in win10 guest, but it won=E2=80=99t load pro=
perly.
 
Despite some conf=
iguration requirements (e.g. always use slot 2) that's
all you need.
 
> &n=
bsp;  2. Tried adding the hd audio device as well, get bluescre=
en with
> =E2=80=9Cmemory_management=E2=80=9D sto=
p code in guest
 
Shouldn'=
t happen but seems like another issue.
 
>    3. Tried adding a gop rom from=

> https://github.com/patmagauran/i915ovmfPkg - host crashed and got=

> error below
 
A GOP is only required for pre OS graphics (e.g. grub menu=
).
 
I highly recommend us=
ing the GOP from your host system to avoid damages
o=
r strange issues due to incompatibilities. Unfortunately, it's not
always possible to easily extract the GOP from your host s=
ystem. Even
Intel recommends to ask your board manuf=
acturer:
https://projec=
tacrn.github.io/3.2/tutorials/gpu-passthru.html#enable-the-gvt-d-gop-driv=
er
 
Some alternative =
ways to extract the GOP:
 
On Windows try GPU-Z:
https://nvidia.custhelp.com/app/answers/detail/a_id/4188/~/ex=
tracting-the-geforce-video-bios-rom-file
On Linu=
x try sysfs:
https://pve.proxmox.com/wiki/P=
CI_Passthrough#The_.27romfile.27_option
&nb=
sp;
>  
> I=E2=80=99m at =
a loss what to try next. Any ideas where I=E2=80=99m going wrong?
>  
> # tail /vm/win10ltsc/=
vm-bhyve.log       
<= pre>> Jan 17 00:49:02:  [primary disk: disk0.img]
> Jan 17 00:49:02:  [primary disk dev: file]
> Jan 17 00:49:02: dynamically allocated port 5900 for vnc conn=
ections
> Jan 17 00:49:02: booting
> Jan 17 00:49:02:  [bhyve options: -c 6,sockets=3D1,cores=
=3D3,threads=3D2 -m
> 8G -Hwl bootrom,/usr/local/=
share/uefi-firmware/BHYVE_UEFI.fd -s
> 2:0,passth=
ru,0/2/0,rom=3D/vm/win10ltsc/i915ovmf.rom -U 8dff5c4a-b083-
> 11ee-b2cd-78d004259091 -S]
> Jan 17 =
00:49:02:  [bhyve devices: -s 0,hostbridge -s 31,lpc -s
> 4:0,ahci,hd:/vm/win10ltsc/disk0.img -s 31:6,passthru,0/31/6=
 -s
> 6:0,fbuf,tcp=3D0.0.0.0:5900,w=3D1920,h=3D10=
80 -s 7:0,xhci,tablet]
> Jan 17 00:49:02:  [=
bhyve console: -l com1,/dev/nmdm-win10ltsc.1A]
=
 
You have to use the `-A` flag. There's an open PR =
for vm-bhyve:
htt=
ps://github.com/churchers/vm-bhyve/pull/525/commits/653d01f5b53c0bd8ce707=
137f67617536de8b1c0
 
=
> Jan 17 00:49:02: starting bhyve (run 1)
> Ja=
n 17 00:49:04: bhyve exited with status 134
> Jan=
 17 00:49:04: stopped
>  
> # tail /vm/win10ltsc/bhyve.log   
=
> pci_fbuf: mmap_memseg failed
> Assertion fai=
led: (mr->name =3D=3D memp->name), function unregister_mem,
> file /usr/src/usr.sbin/bhyve/mem.c, line 344.
>  
> # uname -a<=
/pre>
> FreeBSD yellowfellow 14.0-RELEASE-p3 FreeBSD 14.0-RELEASE-=
p3 #0: Mon
> Dec 11 04:56:01 UTC 2023  =
  
> root@amd64-builder=
=2Edaemonology.net:/usr/obj/usr/src/amd64.amd64/sys/G
> ENERIC amd64
 
=
-- 
Kind regards,
Corvin
------=_NextPart_000_0005_B98A84ED.F06AD2C--