From nobody Fri Mar 29 16:32:33 2024 X-Original-To: dev-commits-doc-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4V5mCk0DNhz5DhBB for ; Fri, 29 Mar 2024 16:32:34 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4V5mCj6Tk8z4TDv; Fri, 29 Mar 2024 16:32:33 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1711729953; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=EJ3GoguL2cYp334nkROcS6apV+Zf7wnOcv5qi2z731s=; b=O1cBYWbOg6NsTRhJudaTOTwwczdnfLdBh7t8dWrj4RSUnSQJr/h2UvGkaZ5SBBDHmCMD0U YEuke1GKXZxzBFpWYU0lCVMH02sSBvGuErwWEYTFrmfm3OuCeBvAT6gNV4K3ysI9bNtQZw qgv5WXD7akab2nSrjkS734qtcXAMtvy1HWRnUxLDTGb9glZ01+jp9C+I3YUA64uf+nk8o4 JAng3WAGhQRfWZ1MSIcQZ+5oUUW33E3utVjYq9D4raqBkz9W4BqbDqwNaO3zEJbI7GVWyp J6a07gAuWJ+ytQePHLLbFJ48CWByHearhWNDpcuZsZ8MeTjtq0gGqTbdw+NQaQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1711729953; a=rsa-sha256; cv=none; b=mQMkHD51LKJPbRXfJ+CtOCna+SBT5F1xmrAviZ0fWU3HjWfBGDIEqAQctD4o2M4MEIY4uQ 3e8kYy4htc/LRMnwXzVOugHNhGiX8kWb7tO2SaCi+L6VWeAKM/kY5xtFPV9pDtsccFP8ik oAr7n2eOHZTje8/LLXkrABd+F7X57wCJHxfaDb/3MBnkYlg6kvITDcnsItS8nQnrlH2O1k i1Dfi34lrEPRBcEC6vD1YHz2MecNM5aiVHLFTYvEFIfaGIW0Bs+BXXw14aZb/Ay94Hh6/S Yj/P7vceilY/Q5180o4j4vqqgGelMH7O6otQmC8WLhmw1B35AuoNix57kGA9NA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1711729953; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=EJ3GoguL2cYp334nkROcS6apV+Zf7wnOcv5qi2z731s=; b=oXzf2U6BCiQeeNlKXHRLp4FU0+qJe8goa0JQHAtIDP89mgOHzi2Uk09C5o81J8c5NdioZC Aq2t7eCSPJUNn6es2yhPTM9pk4dYn15Unpv5+aRHDi2zP4O8jH5HP64t/IMSug7YK3K6wy CPA1RmPPjHeo/uYU9AnGEKDX6mDIHz6boaNQWaJxiPEjy/+6+lYZeWSXVJveM8h0T0o7Ze J3DoHtjm0hh0XKsuC4XOmhgqWEuHQzW87pBi+gxZLWHEG2jj8hcDdfpzPfm6UpjXrr1Kxe duu377/MjgfGYL4uLkzzxV/KvPe8y5ZIW7ixsdHDq4L93ezKGgCcdZDm5cTQrw== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4V5mCj65BgzJ3V; Fri, 29 Mar 2024 16:32:33 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.17.1/8.17.1) with ESMTP id 42TGWXV9013496; Fri, 29 Mar 2024 16:32:33 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.17.1/8.17.1/Submit) id 42TGWXaA013493; Fri, 29 Mar 2024 16:32:33 GMT (envelope-from git) Date: Fri, 29 Mar 2024 16:32:33 GMT Message-Id: <202403291632.42TGWXaA013493@gitrepo.freebsd.org> To: doc-committers@FreeBSD.org, dev-commits-doc-all@FreeBSD.org From: Lorenzo Salvadore Subject: git: 2fc0dfb4ba - main - Status/2024Q1/bhyve.adoc: Add report List-Id: Commit messages for all branches of the doc repository List-Archive: https://lists.freebsd.org/archives/dev-commits-doc-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-doc-all@freebsd.org X-BeenThere: dev-commits-doc-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: salvadore X-Git-Repository: doc X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 2fc0dfb4ba3b057545fb5643d035552667fc7ee5 Auto-Submitted: auto-generated The branch main has been updated by salvadore: URL: https://cgit.FreeBSD.org/doc/commit/?id=2fc0dfb4ba3b057545fb5643d035552667fc7ee5 commit 2fc0dfb4ba3b057545fb5643d035552667fc7ee5 Author: Chris Moerz AuthorDate: 2024-03-29 16:32:11 +0000 Commit: Lorenzo Salvadore CommitDate: 2024-03-29 16:32:18 +0000 Status/2024Q1/bhyve.adoc: Add report Reviewed by: status (Pau Amma ) --- .../en/status/report-2024-01-2024-03/bhyve.adoc | 72 ++++++++++++++++++++++ 1 file changed, 72 insertions(+) diff --git a/website/content/en/status/report-2024-01-2024-03/bhyve.adoc b/website/content/en/status/report-2024-01-2024-03/bhyve.adoc new file mode 100644 index 0000000000..7e94c66526 --- /dev/null +++ b/website/content/en/status/report-2024-01-2024-03/bhyve.adoc @@ -0,0 +1,72 @@ +=== Bhyve Improvements + +Links: + +link:https://callfortesting.org/[bhyve production users calls] URL: link:https://callfortesting.org[] +link:https://wiki.freebsd.org/EnterpriseWorkingGroup[FreeBSD Wiki - Enterprise Working Group] URL: link:https://wiki.freebsd.org/EnterpriseWorkingGroup[] + +link:https://wiki.freebsd.org/ChrisMoerz/bhyve_management[FreeBSD Wiki - EWG - bhyve and jails management tooling] URL: link:https://wiki.freebsd.org/ChrisMoerz/bhyve_management[] + +link:http://static.bultmann.eu/s6-talk/#(1)[Jan Bramkamp's work on s6rc] URL:http://static.bultmann.eu/s6-talk/#(1)[] + +link:https://github.com/christian-moerz/vmstated[vmstated on Github] URL: link:https://github.com/christian-moerz/vmstated[] + +link:https://www.youtube.com/watch?v=f60NCrunXyw[YouTube - vmstated explained] URL: link:https://www.youtube.com/watch?v=f60NCrunXyw[] + +Contact: Chris Moerz + +==== Bhyve I/O Performance Measurements + + +Participants of the weekly bhyve production users calls recently discussed bhyve's I/O performance. +Various ways of measuring and comparing were brought up, however it was quickly clear that there is currently no formal analysis and report on this. +So, we started this effort in the hopes of better understanding the various impacts of configuration options for a guest on its I/O performance. +We created a set of shell scripts that harness a FreeBSD guest for running package:benchmarks/fio[] I/O performance measurements under various configurations. +This allows us to compare multiple criteria like bandwidth, latency, IOPS, and more. + +So far, we are testing for + +* different storage backends (i.e. ahci-hd, nvme, virtio-blk) +* different memory settings +* different CPU pinning options +* different block sizes for the backing storage +* different block sizes for accessing virtual disks + +We are also pitting results for different CPU manufacturers against each other and contrasting guest vs host performance to better understand the performance impact of virtualization. + +We plan to continue discussing our results during Michael Dexter's weekly bhyve production users call - come join us if you are interested. +We also hope to be able to present the results at EuroBSDCon in Q3. + +==== Bhyve Virtual Machine Tooling + +Last year, Greg Wallace at the FreeBSD Foundation founded the Enterprise Working Group with the specific goal of addressing pain points of Enterprise users of FreeBSD. +One of the work groups that emerged clustered around bhyve and jails management tooling. +After collecting a set of desired features and functionality, one overarching key point for bhyve emerged: the desire to have configuration concepts and tooling for bhyve like the ones available for jails. + +While other desirable features were identified as well, i.e. TPM software emulation and snapshot/restore/host-migration, the conceptual tooling question won over those due to the lower degree of complexity and its clarity on goal and the path on how to take steps towards it. + +Technically, this means working out existing gaps around process supervision and virtual machine state management. +First steps were taken by experimenting with existing frameworks (i.e. s6rc work by Jan Bramkamp) and eventually -- through discussions in the weekly bhyve production user's calls (organized by Michael Dexter) -- this led to a proof-of-concept implementation of "vmstated". + +Started as an experiment to better understand the problem space of process supervision and virtual machine state handling, vmstated is constructed of a daemon and vmstatedctl management utility. +It is built with base-only tooling and libraries and leverages FreeBSD specific constructs like kqueue to minimize its resource impact. + +vmstated is configured via a UCL configuration file (similar to [.filename]#jails.conf#) and -- in combination with a man:bhyve_config|5] configuration file -- already provides highest flexibility in configuring virtual machines. +vmstatedctl provides a jail-like command set to start, stop, and retrieve status information about guests. +State transitions can easily be hooked via shell scripts and allow running additional commands for network or storage set up and tear down when relevant state changes occur. + +An initial release is already in ports as package:sysutils/vmstated[] and updates are pending commit; however, the newest version can be found on GitHub. +We are considering expanding the work; we would also like to invite anyone interested to join us in this work! +Patches, suggestions, feedback, etc. are all very much welcome! + +If you want to know more about our work, come join us at one of Michael Dexter's weekly bhyve production users calls or reach me mailto:freebsd@ny-central.org[by email]. + +==== Documentation === + +We managed to update a few parts of the Handbook and Porter's Handbook (thanks to mailto:emaste@FreeBSD.org[Ed Maste], mailto:jrm@FreeBSD.org[Joseph Mingrone], mailto:pauamma@gundo.com[Pau Amma], and mailto:rgrimes@FreeBSD.org[rgrimes@FreeBSD.org]): + +* several improvements and expansions to the virtualization chapter in the FreeBSD Handbook +** using a man:bhyve_config|5] configuration file +** jailing bhyve +** experimental snapshot and restore feature +** setting up a Windows guest +* we also have a review (link:https://reviews.freebsd.org/D43940[D43940]) up for an initial step to improving the bhyve man page +** this was intentionally started with a structural update first to separate the many `-s` flag options +** once this lands, we can move to a more widespread update to the overall content + +Feedback is obviously very welcome -- on the existing content as well as any additional content we should be looking into!