Re: bhyve/passthru for Intel dGPU (ARC A380)?

Hi Peter,

unfortunately, this iGPU stuff is a bit messy. For some reason, Intel is unable
to design them in a platform independent way. That's why bhyve has to probe for
it and apply some quirks to it in it's GVT-d code. It's not expected that those
quirks are required for dedicated GPUs. So, please try to skip the GVT-d check
you've already found.

If you want to pass the option rom to the guest, you can use the rom option of
passthru devices:

-s 1/2/3,passthru,1/2/3,rom=/path/to/rom


--
Kind regards,
Corvin

On Sun, 2025-01-05 at 10:38 +0000, Peter Wood wrote:
> Hello Mario,
> 
> Thanks for the response. Interesting, is the expectation that an dGPU ARC
> would work with the gvt-d code in place? Which would reenforce I'm fighting
> the motherboard/BIOS.
> 
> The BIOS is infact in CSM, as you've suggested, but for reason - though this
> isn't a -virtualization problem, though I'll explain it for any future reader:
> 
> My experience of the ROMED8-2T is when in pure UEFI after the FreeBSD loader
> starts the framebuffer console there are no further updates via the AST2500 if
> the Intel ARC is present, I suspect the framebuffer is being started on the
> Intel ARC - but I don't have a monitor capable of checking that near the
> server.
> 
> The BIOS (3.5 and 3.8 checked) only offers configuration of the preferred
> graphical output if the BIOS is CSM, and even then the framebuffer only seems
> to stay on the AST2500 if the video option rom is set to legacy mode.
> 
> For pure EFI, I haven't dug into loader yet, but what I have observed is that
> the ROMED8-2T does not output the EFI variable ConOut, which having a quick
> read would probably guide the loader where to push the framebuffer. I don't
> know if it's possible for me to write that var to EFI, but may be worth
> looking into.
> 
> As an aside, with CSM (video legacy option rom) I did actually patch bhyve to
> remove the gvt-d check, and the linux guest did boot with the passthru - I
> assume with the card in legacy mode... the linux kernel was happy with it
> until it tried to read the option rom, which I haven't exported yet - but I'd
> prefer to fix this the correct way if possible.
> 
> P.
> 
> 
> On Sat, 4 Jan 2025 at 20:34, Mario Marietto <marietto2008@gmail.com> wrote:
> > Hi Peter,
> >  
> > Please make sure your GPU is booted in UEFI and not CSM mode. This can be
> > changed in the host BIOS.
> > Cheers. 
> > 
> > 
> > On Sat, Jan 4, 2025 at 9:24 PM Peter Wood <peter@alastria.net> wrote:
> > > Happy new year all.
> > > 
> > > I've been using bhyve happily for a year or two now, multiple machines
> > > with VT-d running happily passing SAS cards and USB cards into VMs.
> > > 
> > > I've reached the point where I want to pass a GPU in for accelerated
> > > encoding/decoding/etc (scrypted, tdarr, jellyfin). I picked up an Intel
> > > ARC A380, as it's encoders/decoders are pretty decent for my use case -
> > > I'd also seen that there had been success with people using the iGPU's in
> > > Intel CPUs successfully.
> > > 
> > > Unfortunately after attaching the GPUs PCI device to ppt, and attempting
> > > to start a VM with it attached, I'm greated by an error that seems to
> > > focus on iGPUs?
> > > 
> > > /usr/sbin/bhyve -A -H -w -u -S -c 2 -m 8G -l com1,/dev/nmdm202B -l
> > > bootrom,/usr/local/share/uefi-
> > > firmware/BHYVE_UEFI.fd,/usr/local/var/cache/bmd/k8s-worker-3.vars -s
> > > 0,hostbridge -s 1,lpc -s 2,nvme,/dev/zvol/vm/k8s-worker-3 -s 3,ahci-
> > > cd,/mnt/vm/isos/ubuntu-24.04-live-server-amd64.iso -s 5,virtio-net,tap9 -s
> > > 4,passthru,4/0/0 -s 6,fbuf,tcp=0.0.0.0:6007,w=1280,h=720,vga=io -s
> > > 7,xhci,tablet test
> > > bhyve: Warning: Unable to reuse host address of Graphics Stolen Memory.
> > > GPU passthrough might not work properly.
> > > bhyve: gvt_d_setup_opregion: Invalid OpRegion signature
> > > bhyve: gvt_d_init: Unable to setup OpRegion
> > > Device emulation initialization error: No such file or directory
> > > 
> > > The machine is an AMD EPYC 7343 on a ASRock ROMED8-2T, it has a dedicated
> > > onboard GPU (attached to the BMC), which the BIOS is configured to
> > > encourage operating systems to use as the primary display - and sure
> > > enough the loader and BSD console are presented through the BMCs ASPEED
> > > AST2500.
> > > 
> > > Digging through the freebsd source tree, it appears that that pci_gvt-d.c
> > > is responsible for this, if it's Intel and a Display then attempt the set
> > > up of graphics memory.
> > > https://github.com/freebsd/freebsd-src/blob/b662ca1d6cd82044c6cb79075e1830b97594bef3/usr.sbin/bhyve/amd64/pci_gvt-d.c#L44
> > > 
> > > Has anyone experimented with this? Can I just patch this out, rebuild
> > > bhyve and expect a chance of success?
> > > 
> > > Cheers,
> > > 
> > > P.
> > > -- 
> > > Peter Wood
> > > peter@alastria.net
> > > 
> > 
> > 
> > -- 
> > Mario.
> 
> 
> -- 
> Peter Wood
> peter@alastria.net
> 

-- 
Kind regards,
Corvin