From nobody Tue Jul 16 20:15:57 2024 X-Original-To: virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WNr1B4Sjgz5Q9DF for ; Tue, 16 Jul 2024 20:15:58 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WNr1B3y8sz48Wk for ; Tue, 16 Jul 2024 20:15:58 +0000 (UTC) (envelope-from kevans@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1721160958; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Q1TFDZCQqZF4DuM82765fnwLPVsgDkezsAEV0/sgy1g=; b=nX5WTlc6V3v0JnIUt/NhFb5rmEXamE1o/94f8VnShHyzcV8WWVSxynSOpncNddVdGF5Iaw QdEBFyRR6T+OiRNIkgRbO0QixouU0qnmp7CBWnJYqdHywomT7F9aTz5NNyoJo3KeGTdIGw V3ME6ATbGKNBtSN9pvbqCNmPUBymrJq63LJkVMGEtKvrZqxABqyLfazsYSGlESjOLvIuO/ Pbn9YEGi8poiP58wdZ5I43YErgAGBGKT9MBqLi04DQXtce1vMMXAoMsafmNfjQ5+e/3s9D U5xPdKieNPk9xiOTlLV/D9BprhQIvDMxmtGJJ09m3CBk3w9Wg6yVEGyptexKcw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1721160958; a=rsa-sha256; cv=none; b=RAAO0o4e3rOYBW5bmgyhmhk9N1kmEfVWB1FX5gjth7uDDf7olWmD6p7u+okNMUwifgAQee AObxNtEdU3OWfvSfnjII77sT+8aEGB5m5CL/EzDEw7KNL5JiroUZb1qZIuAQP0+VZI44PA AoKMJqivxl0KNcpq5O2a2cTyPtB87o69n6lqn27xAw0CSmQZ982yNx1PN7XIcpvAqh+hj0 gnEbrErdqqdsqwu+Qgo796KpJj4+/Nn8dwbQklfYQfdl/ty3NGV3TEawGZNOUXwugKxExb n+F1lonKrZ1T6XT4wdWfkHV7n+WdLOZ4ZUtQpJvr8GOaevAvRzdMvs1TdZcB6w== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1721160958; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Q1TFDZCQqZF4DuM82765fnwLPVsgDkezsAEV0/sgy1g=; b=aVzoLMypUl0qNjLv72wsAjULhJji8X4FRtbpRELTq/ETBSAP2Sv56o+iaze5169c3+Qxag VBM9drPdwidgHcrZ8B+e5qu/25PTztoEjaIsMGRfAacBX4YjqElMPrd3Wtwwh8KV/5A079 eCCAgF2wNetERCpE/9bpTgcF9sX66j1uRV0fyJ8qw3VJ+HGGAIMvl1mNX1w7pXb9NTEp3r v7YMSe2Zn2dVWe5RmfdIwMlPkZb6ItihTiAH8u2ONDzu/iD20XYqbertWdqSrO5PoImt4r Ta35QYq6r/dU6//fVTMUualPnE5t75dEAoUu3gejS/4z/D1uEfQTkrGqriDuMg== Received: from [10.9.4.95] (unknown [209.182.120.176]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) (Authenticated sender: kevans/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4WNr1B2JGczhrl for ; Tue, 16 Jul 2024 20:15:58 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Message-ID: <16dfe6b3-2ef1-4859-a5b0-a36efe452804@FreeBSD.org> Date: Tue, 16 Jul 2024 15:15:57 -0500 List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: freebsd-virtualization@freebsd.org Sender: owner-freebsd-virtualization@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: nmdm issues with bhyve To: virtualization@freebsd.org References: <20240716153851.60c1ea61@PolarianBSD> Content-Language: en-US From: Kyle Evans In-Reply-To: <20240716153851.60c1ea61@PolarianBSD> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit On 7/16/24 09:38, Polarian wrote: > Hello, > > Over the last few days I have been trying to get nmdm working with > bhyve. I have discussed it within #bhyve over on libera.chat however > none of the suggestions so far have fixed the problem, therefore I am > bringing it to the mailing list for further support. > > A rundown what I have done so far, I have a shell script with the > following contents: > > #!/bin/sh > > bhyve -c 1 -m 1G -u -H \ > -s 0,amd_hostbridge \ > -s 3,ahci-hd,/path/to/install.img \ > -s 4,virtio-blk,/dev/zvol/zpool-storage/dns \ > -s 5,virtio-net,tap0 \ > -s 31,lpc -l com1,/dev/nmdm3A \ > -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \ > dns > > To verify the virtual machine was in fact booting, I did test stdio, > and the install medium for OpenBSD does successfully boot. > > The problem, well there is many. Following the FAQ entry [1], the > following flag is suggested: > > -s 31,lpc -l com1,nmdm0A > > (where 0 can be substituted to prevent collisions) > > Enter problem #1, when I try this by executing the script (with doas, > doas sh dns.sh) I get the following: > > Unable to initialize backend 'nmdm3A' for LPC device com1 > Device emulation initialization error: No such file or directory > > Now, both vmm and nmdm have been loaded, checked with the following > commands: > > kldstat | grep vmm > kldstat | grep nmdm > > And these have been entered into /boot/loader.conf see below: > > vmm_load="YES" > nmdm_load="YES" > > So they have been loaded into the kernel. > > When looking into the problem I found an article on the FreeBSD forums > which has the same error [2], which would suggest the example in the > FAQ is either wrong, or deprecated and no longer works. > > The solution from this article is to ensure to specify the /dev path, as > seen in the full script at the beginning of the email, /dev/nmdm0A, > this DOES work and the vm does startup. > > Problem #2, the device doesn't show up, even though the vm is, in fact, > running. > > ls /dev | grep nmdm > > The above returns nothing, there is no nmdm device. I have tested it to > ensure nmdm is in fact working, using cu to open both A and B and > verifying that the data is being passes between them. So nmdm is > working! > We did some light debugging in #freebsd on this; bhyve(8) is opening the nmdm pair as it should and that works fine, but I guess at some point (perhaps right after handoff to the OpenBSD kernel) it's getting closed so the device isn't around anymore by the time they try to observe it above. They attempted a run with stdio in otherwise the same configuration, and it cut off right after: booting hd0a:/7.5/amd64/bsd.rd: 4076463+1688576+3891240+0+708608 [109+464016+317541]=0xaa40e8 entry point at 0x1001000 wrmsr to register 0xc0011029(0x3) on vcpu 0 Thanks, Kyle Evans