From nobody Sat Sep 14 14:18:53 2024 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4X5YGf4t1jz5WSLx for ; Sat, 14 Sep 2024 14:19:54 +0000 (UTC) (envelope-from chuck@tuffli.net) Received: from fhigh5-smtp.messagingengine.com (fhigh5-smtp.messagingengine.com [103.168.172.156]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4X5YGd5jJXz4YVq for ; Sat, 14 Sep 2024 14:19:53 +0000 (UTC) (envelope-from chuck@tuffli.net) Authentication-Results: mx1.freebsd.org; none Received: from phl-compute-12.internal (phl-compute-12.phl.internal [10.202.2.52]) by mailfhigh.phl.internal (Postfix) with ESMTP id 413D311401A0; Sat, 14 Sep 2024 10:19:52 -0400 (EDT) Received: from phl-imap-11 ([10.202.2.101]) by phl-compute-12.internal (MEProxy); Sat, 14 Sep 2024 10:19:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tuffli.net; h=cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm1; t=1726323592; x=1726409992; bh=MG+a5XrfZf oEPWtdzPHiiy0ZJkrJSqQuI1JFwfTx9IE=; b=Lbmz9B2UWb6IskvdilvAjyar+R 4A5I/lDP9ux0QHFc/XqfzNSputdxoZ1NSeDBHhsGM9YXzbb2SzkSGSb7RBLIS6TN frT8xnoQ8pqMfOpNxL8USUB3+neF9ardiNFu718ZE9oP4Yv2Bxge1Uf+I6EirtgA /NsYA4PpLv4TYgPWAIRFCJSZrMv2xRoxSVJUW3H/pRqzk0bu4I0Rl0JFcFZ/3CIN Q+wCbGL8uFz/QWgikFAd8T5OpF9J+ntvm8YJGuh8J65zTInA/zVjaP4iYiF6IEsT gPUinnJaDX0+4Dr5qrzim9QPfQ97kl3rf19mNB+ubk03xKdn1iGsl/FMmDQQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1726323592; x=1726409992; bh=MG+a5XrfZfoEPWtdzPHiiy0ZJkrJ SqQuI1JFwfTx9IE=; b=p/+I33MdJR896YXmZ1l7WYBctdVqABizkJ8cclGr66ur lV0l2KhoIdWVbwfit9otCo3NP+VM6qIfV1sMccva2NWUMMhKZjHR1J9QBRyPbGMJ cBWRDEivlJGI6/3hVnxwVF91fJdKA0Cjm2YK1bVI4dzDiiYzot94/waUVtimFevO iAeGXD3MrqFbVZdMbugxWXxgw3scME1nPyFDgfM3RFT8VUybbuwvUECwM3Wqv/Op 5he2e5pqTOVCMMTi7WofKLkTWPKoFE5wD60MVm0pyfYxu9FBoxhT5UdIEi8C0voV 2o4biWjMK83frsyz7HITBs2tGXPqJfvdJzm9fOF43Q== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeftddrudektddgjeehucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggvpdfu rfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnh htshculddquddttddmnecujfgurhepofggfffhvffkjghfufgtsegrtderreertdejnecu hfhrohhmpedfvehhuhgtkhcuvfhufhhflhhifdcuoegthhhutghksehtuhhffhhlihdrnh gvtheqnecuggftrfgrthhtvghrnhepveejudevheejgfehgefhhfevuedtueejjeelteel ffeutdettdekkeeuvdfgffejnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpe hmrghilhhfrhhomheptghhuhgtkhesthhufhhflhhirdhnvghtpdhnsggprhgtphhtthho pedvpdhmohguvgepshhmthhpohhuthdprhgtphhtthhopehvohhiugesfhdqmhdrfhhmpd hrtghpthhtohepfhhrvggvsghsugdqvhhirhhtuhgrlhhiiigrthhiohhnsehfrhgvvggs shgurdhorhhg X-ME-Proxy: Feedback-ID: ib6f94606:Fastmail Received: by mailuser.phl.internal (Postfix, from userid 501) id 8E537222006F; Sat, 14 Sep 2024 10:19:51 -0400 (EDT) X-Mailer: MessagingEngine.com Webmail Interface List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: freebsd-virtualization@freebsd.org Sender: owner-freebsd-virtualization@FreeBSD.org MIME-Version: 1.0 Date: Sat, 14 Sep 2024 07:18:53 -0700 From: "Chuck Tuffli" To: void , "FreeBSD virtualization" Message-Id: <484d66f9-3f61-4512-b7e7-6142c79fd04c@app.fastmail.com> In-Reply-To: References: Subject: Re: nvme virtualisation string for bhyve Content-Type: multipart/alternative; boundary=72ff171a59e94fc2b4713140065a723a X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:209242, ipnet:103.168.172.0/24, country:US] X-Rspamd-Queue-Id: 4X5YGd5jJXz4YVq --72ff171a59e94fc2b4713140065a723a Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On Sat, Sep 14, 2024, at 6:14 AM, void wrote: > ... > I usually start my vms with a bhyve variable for the disk > like: >=20 > -s 4,virtio-blk,/dev/zvol/ssd/vm2 \ >=20 > The man page for bhyve says this: >=20 > ### NVMe device backends > ### =E2=80=A2 devpath[,maxq=3D#][,qsz=3D#][,ioslots=3D#][,sect= sz=3D#][,ser=3D#][,eui64=3D#][,dsm=3Dopt] >=20 > How do I get this information? Like maxq qsg ioslots sectorz ser eui64= dsm >=20 > The ssds are: >=20 > nvme0: > nvme1: If you want the guest to see their storage as an NVMe device, you would = replace "virtio-blk" with "nvme" in your device specification above (i.e= ., -s 4,nvme,...). The bhyve NVMe emulation code will pick sensible valu= es for the above device characteristics like the max number of supported= queues (a.k.a., maxq). If you want to override the values bhyve picks, = you can add those key-value pairs after the device name. Note that the emulated device the guest VM sees is entirely separate fro= m the devices providing the storage to the host OS. As an example, the g= uest can have an NVMe drive backed by a zpool on the host comprised of S= ATA or SCSI drives. So if the question was, "how do I determine maxq, qsz, etc. from the hos= t drive information?", the answer is, you wouldn't. --chuck --72ff171a59e94fc2b4713140065a723a Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable
On Sat, Sep 14,= 2024, at 6:14 AM, void wrote:
...
I usually start my vms with a bhyv= e variable for the disk
like:

-s 4,virtio-blk,/dev/zvol/ssd/vm2 \

The ma= n page for bhyve says this:

###  NVMe = device backends
###       =E2= =80=A2   devpath[,maxq=3D#][,qsz=3D#][,ioslots=3D#][,sectsz=3D= #][,ser=3D#][,eui64=3D#][,dsm=3Dopt]

How do= I get this information? Like maxq qsg ioslots sectorz ser eui64 dsm
=

The ssds are:

nvm= e0: <Intel DC PC4500>
nvme1: <DC P3700 SSD>

If you want the guest to see the= ir storage as an NVMe device, you would replace "virtio-blk" with "nvme"= in your device specification above (i.e., -s 4,nvme,...). The bhyve NVM= e emulation code will pick sensible values for the above device characte= ristics like the max number of supported queues (a.k.a., maxq). If you w= ant to override the values bhyve picks, you can add those key-value pair= s after the device name.

Note that the emul= ated device the guest VM sees is entirely separate from the devices prov= iding the storage to the host OS. As an example, the guest can have an N= VMe drive backed by a zpool on the host comprised of SATA or SCSI drives= .

So if the question was, "how do I determi= ne maxq, qsz, etc. from the host drive information?", the answer is, you= wouldn't.

--chuck
--72ff171a59e94fc2b4713140065a723a--