Re: nmdm issues with bhyve

On 7/16/24 09:38, Polarian wrote:
> Hello,
> 
> Over the last few days I have been trying to get nmdm working with
> bhyve. I have discussed it within #bhyve over on libera.chat however
> none of the suggestions so far have fixed the problem, therefore I am
> bringing it to the mailing list for further support.
> 
> A rundown what I have done so far, I have a shell script with the
> following contents:
> 
> #!/bin/sh
> 
> bhyve -c 1 -m 1G -u -H \
>          -s 0,amd_hostbridge \
>          -s 3,ahci-hd,/path/to/install.img \
>          -s 4,virtio-blk,/dev/zvol/zpool-storage/dns \
>          -s 5,virtio-net,tap0 \
>          -s 31,lpc -l com1,/dev/nmdm3A \
>          -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
>          dns
> 
> To verify the virtual machine was in fact booting, I did test stdio,
> and the install medium for OpenBSD does successfully boot.
> 
> The problem, well there is many. Following the FAQ entry [1], the
> following flag is suggested:
> 
> -s 31,lpc -l com1,nmdm0A
> 
> (where 0 can be substituted to prevent collisions)
> 
> Enter problem #1, when I try this by executing the script (with doas,
> doas sh dns.sh) I get the following:
> 
> Unable to initialize backend 'nmdm3A' for LPC device com1
> Device emulation initialization error: No such file or directory
> 
> Now, both vmm and nmdm have been loaded, checked with the following
> commands:
> 
> kldstat | grep vmm
> kldstat | grep nmdm
> 
> And these have been entered into /boot/loader.conf see below:
> 
> vmm_load="YES"
> nmdm_load="YES"
> 
> So they have been loaded into the kernel.
> 
> When looking into the problem I found an article on the FreeBSD forums
> which has the same error [2], which would suggest the example in the
> FAQ is either wrong, or deprecated and no longer works.
> 
> The solution from this article is to ensure to specify the /dev path, as
> seen in the full script at the beginning of the email, /dev/nmdm0A,
> this DOES work and the vm does startup.
> 
> Problem #2, the device doesn't show up, even though the vm is, in fact,
> running.
> 
> ls /dev | grep nmdm
> 
> The above returns nothing, there is no nmdm device. I have tested it to
> ensure nmdm is in fact working, using cu to open both A and B and
> verifying that the data is being passes between them. So nmdm is
> working!
> 

We did some light debugging in #freebsd on this; bhyve(8) is opening the 
nmdm pair as it should and that works fine, but I guess at some point 
(perhaps right after handoff to the OpenBSD kernel) it's getting closed 
so the device isn't around anymore by the time they try to observe it 
above.  They attempted a run with stdio in otherwise the same 
configuration, and it cut off right after:

booting hd0a:/7.5/amd64/bsd.rd: 4076463+1688576+3891240+0+708608 
[109+464016+317541]=0xaa40e8
entry point at 0x1001000
wrmsr to register 0xc0011029(0x3) on vcpu 0

Thanks,

Kyle Evans