Re: bhyve/passthru for Intel dGPU (ARC A380)?

Hello Mario,

Thanks for the response. Interesting, is the expectation that an dGPU ARC
would work with the gvt-d code in place? Which would reenforce I'm fighting
the motherboard/BIOS.

The BIOS is infact in CSM, as you've suggested, but for reason - though
this isn't a -virtualization problem, though I'll explain it for any future
reader:

My experience of the ROMED8-2T is when in pure UEFI after the FreeBSD
loader starts the framebuffer console there are no further updates via the
AST2500 if the Intel ARC is present, I suspect the framebuffer is being
started on the Intel ARC - but I don't have a monitor capable of checking
that near the server.

The BIOS (3.5 and 3.8 checked) only offers configuration of the preferred
graphical output if the BIOS is CSM, and even then the framebuffer only
seems to stay on the AST2500 if the video option rom is set to legacy mode.

For pure EFI, I haven't dug into loader yet, but what I have observed is
that the ROMED8-2T does not output the EFI variable ConOut, which having a
quick read would probably guide the loader where to push the framebuffer. I
don't know if it's possible for me to write that var to EFI, but may be
worth looking into.

As an aside, with CSM (video legacy option rom) I did actually patch bhyve
to remove the gvt-d check, and the linux guest did boot with the passthru -
I assume with the card in legacy mode... the linux kernel was happy with it
until it tried to read the option rom, which I haven't exported yet - but
I'd prefer to fix this the correct way if possible.

P.


On Sat, 4 Jan 2025 at 20:34, Mario Marietto <marietto2008@gmail.com> wrote:

> Hi Peter,
>
>
>
> Please make sure your GPU is booted in UEFI and not CSM mode. This can be
> changed in the host BIOS.
>
> Cheers.
>
>
>
> On Sat, Jan 4, 2025 at 9:24 PM Peter Wood <peter@alastria.net> wrote:
>
>> Happy new year all.
>>
>> I've been using bhyve happily for a year or two now, multiple machines
>> with VT-d running happily passing SAS cards and USB cards into VMs.
>>
>> I've reached the point where I want to pass a GPU in for accelerated
>> encoding/decoding/etc (scrypted, tdarr, jellyfin). I picked up an Intel ARC
>> A380, as it's encoders/decoders are pretty decent for my use case - I'd
>> also seen that there had been success with people using the iGPU's in Intel
>> CPUs successfully.
>>
>> Unfortunately after attaching the GPUs PCI device to ppt, and attempting
>> to start a VM with it attached, I'm greated by an error that seems to focus
>> on iGPUs?
>>
>> /usr/sbin/bhyve -A -H -w -u -S -c 2 -m 8G -l com1,/dev/nmdm202B -l
>> bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd,/usr/local/var/cache/bmd/k8s-worker-3.vars
>> -s 0,hostbridge -s 1,lpc -s 2,nvme,/dev/zvol/vm/k8s-worker-3 -s
>> 3,ahci-cd,/mnt/vm/isos/ubuntu-24.04-live-server-amd64.iso -s
>> 5,virtio-net,tap9 -s 4,passthru,4/0/0 -s 6,fbuf,tcp=0.0.0.0:6007,w=1280,h=720,vga=io
>> -s 7,xhci,tablet test
>> bhyve: Warning: Unable to reuse host address of Graphics Stolen Memory.
>> GPU passthrough might not work properly.
>> bhyve: gvt_d_setup_opregion: Invalid OpRegion signature
>> bhyve: gvt_d_init: Unable to setup OpRegion
>> Device emulation initialization error: No such file or directory
>>
>> The machine is an AMD EPYC 7343 on a ASRock ROMED8-2T, it has a dedicated
>> onboard GPU (attached to the BMC), which the BIOS is configured to
>> encourage operating systems to use as the primary display - and sure enough
>> the loader and BSD console are presented through the BMCs ASPEED AST2500.
>>
>> Digging through the freebsd source tree, it appears that that pci_gvt-d.c
>> is responsible for this, if it's Intel and a Display then attempt the set
>> up of graphics memory.
>>
>> https://github.com/freebsd/freebsd-src/blob/b662ca1d6cd82044c6cb79075e1830b97594bef3/usr.sbin/bhyve/amd64/pci_gvt-d.c#L44
>>
>> Has anyone experimented with this? Can I just patch this out, rebuild
>> bhyve and expect a chance of success?
>>
>> Cheers,
>>
>> P.
>> --
>> *Peter Wood*
>> peter@alastria.net
>>
>>
>
> --
> Mario.
>


-- 
*Peter Wood*
peter@alastria.net