Re: bhyve regression (head): windows VMs failing with error 0xc000021a

On 25/10/24 22:49, Mark Johnston wrote:
> On Fri, Oct 25, 2024 at 09:24:13PM +0200, Guido Falsi wrote:
>> Hi,
>>
>> I've recently updated my current machines to git commit
>> 525a177c165740fc697df3de5b92e58b3b41477c (Sun Oct 20 22:43:41 2024 -0800)
>> and just have Windows 10 VMs fail to start in bhyve with the error in the
>> subject.
>>
>> I've been unable to recover them with usual tricks (automatic recovery,
>> chkdsk, and other tools provided by the OS). Looks like the machine fails to
>> read C: after boot.
>>
>> These machines were working fine before the update, so my suspect is that
>> some recent change in bhyve is causing the issue and the VMs would be
>> otherwise fine.
>>
>> The VMs have their filesystems in compressed zvols.
>>
>> Anyone has an idea or can point to some change I can test reverting?
> 
> Just a guess, but you might try adding "-o pci.enable_bars=true" to the
> bhyve command line arguments

Tried but it looks like it made no difference.

> 
>> I an also try bisecting, I'd guess the issue is quite recent.
> 
> Which revision did you update from?

I updated from git commit 450a6690f557493bd33d8f3666b22ddc5150703b (Thu 
Sep 19 11:49:40 2024 -0500)

In the while I noticed some commits to TPM emulation/passthorugh, maybe 
they're related?

> 
>> Thanks in advance for any advice.
>>
>> Obviously if any further info is needed just ask.
> 
> What command line arguments are passed to bhyve?  What boot firmware are
> you using?

I'm using vm-bhyve, from its log the options should have been:

-c 4,sockets=1,cores=2,threads=2 -m 4G -AHPw -l 
bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd -o 
pci.enable_bars=true -U xxx -s 0,hostbridge -s 31,lpc -s 
4:0,ahci,hd:/dev/zvol/zroot/bhyve/W64/disk0 -s 
5:0,virtio-net,tap1,mac=xxx -s 6:0,fbuf,tcp=127.0.0.1:5900,w=1600,h=900 
-s 7:0,xhci,tablet -s 8:0,hda,play=/dev/dsp1 -l com1,stdio

(replaced MAC and UUID with xxx)

for firmware I'm using bhyve-firmware-1.0_2 from sysutils/bhyve-firmware

-- 
Guido Falsi <mad@madpilot.net>