devctl: Failed to detach pci0:1:0:0: Device busy / devctl: Failed to set pci0:1:0:0 driver to ppt: Device busy

From: Mario Marietto <marietto2008_at_gmail.com>
Date: Sun, 29 Jan 2023 19:07:53 UTC
Hello to everyone.

In some FreeBSD 13.1 machines I have the problem below,in some others I
don't have it. I would like to know what the causes could be and how to fix
it.

root@marietto:/usr/home/marietto # devctl detach pci0:1:0:0

devctl: Failed to detach pci0:1:0:0: Device busy

root@marietto:/usr/home/marietto # devctl detach pci0:2:0:0

devctl: Failed to detach pci0:2:0:0: Device busy

Not even it works if instead of detach them, I try to attach them directly
to the ppt driver :

root@marietto:/usr/home/marietto/bhyve/Files # devctl set driver pci0:2:0:0
ppt
devctl: Failed to set pci0:2:0:0 driver to ppt: Device busy

root@marietto:/usr/home/marietto/bhyve/Files # devctl detach pci0:2:0:0
devctl: Failed to detach pci0:2:0:0: Device busy

Pci addresses 1:0:0 and 2:0:0 belong to the two GPUs that I have on my PC :

vgapci0@pci0:1:0:0:     class=0x030000 rev=0xa1 hdr=0x00 vendor=0x10de
device=0x1c02 subvendor=0x19da subdevice=0x2438
vgapci1@pci0:2:0:0:     class=0x030000 rev=0xa1 hdr=0x00 vendor=0x10de
device=0x1e04 subvendor=0x19da subdevice=0x2503

Actually I have commented this line on /boot/loader.conf,because it makes
no difference if I keep it uncommented or not. It is totally ignored.

#pptdevs="1/0/0 1/0/1 2/0/0 2/0/1 2/0/2 2/0/3"

In every FreeBSD machine I load the nvidia kernel modules adding the line
below to /etc/rc.conf :

kld_list="nvidia nvidia-modeset"

and I have installed the nvidia-driver package on every machine. But as I
said,in some of them I see the error above,in some others I don't see it.
I'm not able to isolate the dysfunctional pattern. The xorg.conf file is
the same for each machine. Can someone help me to troubleshoot the error ?
thanks.

-- 
Mario.