Windows guest hangs with PCI passthru if more than once core

From: <jbo_at_insane.engineer>
Date: Thu, 28 Mar 2024 00:33:39 UTC
Hey guys,

I'm currently dealing with an issue that's almost driving me insane.

Scenario:
  - FreeBSD FreeBSD 14.0-STABLE #5 stable/14-n266935-fdfb8e783c3e: Tue Mar  5 17:41:08 CET 2024
  - Intel Core i7-8086K
  - Windows 11 Pro guest

What I'm observing is that the VM guest generally boots and works fine without any issues.
However, once I passthru a PCIe USB controller card, the VM seems to hang when Windows is booting.
At that point, I can only kill -9 the bhyve process. Anything else just hangs (including bhyvectl --destroy).

If I change the VM configuration to use only 1 CPU core, it boots fine 100% of the time with the passthru.
The same is true if I give it 1 CPU core and 2 threads (boots fine 100% of the time).
But once I give it more than 1 CPU core, it locks up when the passthru is enabled.

Is this a known issue? I have seen [1] but I'm not sure whether it's the same scenario or whether
there are any patches I could apply to get me out of this madness.

I earlier posted here [2].

[1] https://lists.freebsd.org/archives/freebsd-virtualization/2023-January/001080.html
[2] https://forums.freebsd.org/threads/bhyve-vm-stuck-when-passthru-enabled.92854


Best regards,
~ jbo@