From nobody Sun Apr 30 21:41:36 2023 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q8ftk3KNrz48LLQ; Sun, 30 Apr 2023 21:41:50 +0000 (UTC) (envelope-from mgrooms@shrew.net) Received: from mx1.shrew.net (mx1.shrew.net [38.97.5.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q8ftj1lVYz4GWP; Sun, 30 Apr 2023 21:41:49 +0000 (UTC) (envelope-from mgrooms@shrew.net) Authentication-Results: mx1.freebsd.org; dkim=none; spf=pass (mx1.freebsd.org: domain of mgrooms@shrew.net designates 38.97.5.131 as permitted sender) smtp.mailfrom=mgrooms@shrew.net; dmarc=none Received: from mail.shrew.net (mail.shrew.prv [10.24.10.20]) by mx1.shrew.net (8.15.2/8.15.2) with ESMTP id 33ULfgOY067025; Sun, 30 Apr 2023 16:41:42 -0500 (CDT) (envelope-from mgrooms@shrew.net) Received: from [10.22.200.32] (unknown [136.49.230.220]) by mail.shrew.net (Postfix) with ESMTPSA id 645C2192C40; Sun, 30 Apr 2023 16:41:37 -0500 (CDT) Message-ID: Date: Sun, 30 Apr 2023 16:41:36 -0500 List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 Subject: Re: BHYVE_SNAPSHOT Content-Language: en-US To: freebsd-hackers@freebsd.org, freebsd-virtualization@freebsd.org References: From: Matthew Grooms Cc: elenamihailescu22@gmail.com, Mihai Carabas , gusev.vitaliy@gmail.com In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mx1.shrew.net [10.24.10.10]); Sun, 30 Apr 2023 16:41:42 -0500 (CDT) X-Spamd-Result: default: False [1.40 / 15.00]; SUBJ_ALL_CAPS(1.05)[14]; NEURAL_HAM_LONG(-1.00)[-0.998]; NEURAL_SPAM_SHORT(0.86)[0.859]; NEURAL_SPAM_MEDIUM(0.79)[0.789]; R_SPF_ALLOW(-0.20)[+mx]; MIME_GOOD(-0.10)[text/plain]; MLMMJ_DEST(0.00)[freebsd-virtualization@freebsd.org,freebsd-hackers@freebsd.org]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:174, ipnet:38.0.0.0/8, country:US]; R_DKIM_NA(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_FIVE(0.00)[5]; FREEMAIL_CC(0.00)[gmail.com]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; FROM_HAS_DN(0.00)[]; TAGGED_RCPT(0.00)[]; TO_DN_SOME(0.00)[]; DMARC_NA(0.00)[shrew.net]; MID_RHS_MATCH_FROM(0.00)[] X-Rspamd-Queue-Id: 4Q8ftj1lVYz4GWP X-Spamd-Bar: + X-ThisMailContainsUnwantedMimeParts: N On 4/29/23 06:17, void wrote: > Hi, > > Where can I read up about the recently introduced BHYVE_SNAPSHOT ? > > I can only find terse information about what it does (in man 5 src.conf) > but not about how to use it and/or why. This began as sponsored student work with the Politehnica University of Bucharest back in 2016 with an eye toward developing Live MIgration as a feature. The initial bhyve save/restore patch was committed back in 2020 ... https://reviews.freebsd.org/rS360648 Student projects related to bhyve continued until last year, but we gave up due to there being such a massive accumulation of patches with no path to getting them committed. We tried reaching out to project maintainers and the FreeBSD foundation alike for support and feedback, but there seems to be little to no interest. There have also been attempts by companies that rely on these features to and get them improved and committed. Unfortunately they don't appear to be having much luck either ... https://reviews.freebsd.org/D38858#885651 Would you like to see support for VM snapshots in the generic kernel? How about support for saving/restore checkpoints using QCOW2, VMDK via libvdsk? How about support for warm or live migration? How about USB device pass-through? There are experimental patches for all these features that were developed by students at UPB. In a lot of cases, there are open reviews that have been waiting on feedback for ages. Here is a a presentation given by Elena from UPB just last month at the FreeBSD devsumit in Tokyo ... https://wiki.freebsd.org/DevSummit/202303?action=AttachFile&do=view&target=Presentation+-+bhyvecon.pdf The case is quite plain. I'm not sure what the solution is to this problem. I'd love to hear feedback from the community about how I've got this completely wrong and how the course could be corrected. That would be something. -Matthew