From nobody Tue May 02 06:16:15 2023 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q9VG52Z9Hz48Xr7; Tue, 2 May 2023 06:16:29 +0000 (UTC) (envelope-from mgrooms@shrew.net) Received: from mx1.shrew.net (mx1.shrew.net [38.97.5.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q9VG36Nx6z4R1T; Tue, 2 May 2023 06:16:27 +0000 (UTC) (envelope-from mgrooms@shrew.net) Authentication-Results: mx1.freebsd.org; dkim=none; spf=pass (mx1.freebsd.org: domain of mgrooms@shrew.net designates 38.97.5.131 as permitted sender) smtp.mailfrom=mgrooms@shrew.net; dmarc=none Received: from mail.shrew.net (mail.shrew.prv [10.24.10.20]) by mx1.shrew.net (8.15.2/8.15.2) with ESMTP id 3426GKTM090463; Tue, 2 May 2023 01:16:20 -0500 (CDT) (envelope-from mgrooms@shrew.net) Received: from [10.22.200.32] (unknown [136.49.230.220]) by mail.shrew.net (Postfix) with ESMTPSA id A1D37193926; Tue, 2 May 2023 01:16:15 -0500 (CDT) Content-Type: multipart/alternative; boundary="------------PosPzAVxthavBLdS0gbQcyBE" Message-ID: <89c84bd7-a925-02f4-acbe-12c3000e7007@shrew.net> Date: Tue, 2 May 2023 01:16:15 -0500 List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 Subject: Re: BHYVE_SNAPSHOT Content-Language: en-US To: Rob Wing Cc: freebsd-hackers@freebsd.org, freebsd-virtualization@freebsd.org, elenamihailescu22@gmail.com, Mihai Carabas , gusev.vitaliy@gmail.com References: From: Matthew Grooms In-Reply-To: X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mx1.shrew.net [10.24.10.10]); Tue, 02 May 2023 01:16:20 -0500 (CDT) X-Spamd-Result: default: False [-0.57 / 15.00]; SUBJ_ALL_CAPS(1.05)[14]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-0.98)[-0.984]; NEURAL_SPAM_SHORT(0.66)[0.663]; R_SPF_ALLOW(-0.20)[+mx]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; MLMMJ_DEST(0.00)[freebsd-virtualization@freebsd.org,freebsd-hackers@freebsd.org]; ASN(0.00)[asn:174, ipnet:38.0.0.0/8, country:US]; R_DKIM_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_TO(0.00)[gmail.com]; RCVD_TLS_LAST(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; FREEMAIL_CC(0.00)[freebsd.org,gmail.com]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_SOME(0.00)[]; DMARC_NA(0.00)[shrew.net]; TAGGED_RCPT(0.00)[]; RCPT_COUNT_FIVE(0.00)[6]; MID_RHS_MATCH_FROM(0.00)[] X-Rspamd-Queue-Id: 4Q9VG36Nx6z4R1T X-Spamd-Bar: / X-ThisMailContainsUnwantedMimeParts: N This is a multi-part message in MIME format. --------------PosPzAVxthavBLdS0gbQcyBE Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit On 4/30/23 21:31, Rob Wing wrote: > Hey Matthew, > > On Sun, Apr 30, 2023 at 1:41 PM Matthew Grooms wrote: > > > Would you like to see support for VM snapshots in the generic kernel? > > > Is there a review open that addresses the limitations described in the > commit message that brought the snapshot feature in? > https://github.com/freebsd/freebsd-src/commit/483d953a86a2507355f8287c5107dc827a0ff516 Yes. The next set of project goals where not pulled out of thin air. They were selected specifically to address the limitations that prevented snapshots from being in the mainline kernel after the initial commit. That's why patches for AMD CPU, Multiple Devices ( >1 of the same type ), Capsicum and JSON file format for snapshots were developed. They were identified as the major per-requisite for lifting conditional compilation. > How about support for warm or live migration? > > > This builds off the snapshot work, right? Seems like it'd make more > sense to address the current limitations of the snapshot code before > extending the functionality off the top of it. > Yup. See above. I appreciate your input, but the goal of live migration was set in 2016 with a prototype first demonstrated in 2018. How long do you suggest a developer wait without review feedback before moving forward out of tree? > There are experimental patches for all these features that were > developed by students at UPB. In a lot of cases, there are open > reviews that have been waiting on feedback for ages. > > > In general, most people don't want to review large experimental patches. Yup. That approach was attempted with the Warm Migration patches. From slide 17 in Elena's presentation: First review opened in 2021: https://reviews.freebsd.org/D28270 5 reviews from 2022 starting with https://reviews.freebsd.org/D34717 (same feature split in multiple parts) A similar request was made recently to Gusev Vitaliy WRT the multiple device support patch which he took ownership of. Thanks for adding feedback to that review BTW. We'll see how that pans out ... https://reviews.freebsd.org/D35590 > > The case is quite plain. I'm not sure what the solution is to this > problem. I'd love to hear feedback from the community about how > I've got > this completely wrong and how the course could be corrected. That > would > be something. > > > My perspective is that it would have been better to focus student > efforts on completing the snapshot feature. By completing the snapshot > feature, I mean getting the code into a state where it's compiled in > by default and no longer considered an experimental feature. > I'm not sure what more to say hear regarding the snapshot feature or what might have been done in the past. We need a solution for the present. If you have any comments related to the follow up reviews submitted by UPB, I'm sure they'd love to hear them. And lastly: I get that FreeBSD is a non paid volunteer project for most. Without the efforts of folks like Peter, Neel, John and others, there would be no bhyve. I'm not saying that they, as project maintainers, should somehow be doing more. We all have limited time to invest, paid work to do and families to feed. I'm asking if there are other developers that might be willing and able to help with reviews? Is there something the FreeBSD foundation can do help out in situations like these? Thanks, -Matthew --------------PosPzAVxthavBLdS0gbQcyBE Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit
On 4/30/23 21:31, Rob Wing wrote:
Hey Matthew,

On Sun, Apr 30, 2023 at 1:41 PM Matthew Grooms <mgrooms@shrew.net> wrote:

Would you like to see support for VM snapshots in the generic kernel?

Is there a review open that addresses the limitations described in the commit message that brought the snapshot feature in? https://github.com/freebsd/freebsd-src/commit/483d953a86a2507355f8287c5107dc827a0ff516

Yes. The next set of project goals where not pulled out of thin air. They were selected specifically to address the limitations that prevented snapshots from being in the mainline kernel after the initial commit. That's why patches for AMD CPU, Multiple Devices ( >1 of the same type ), Capsicum and JSON file format for snapshots were developed. They were identified as the major per-requisite for lifting conditional compilation.

 
How about support for warm or live migration?

This builds off the snapshot work, right? Seems like it'd make more sense to address the current limitations of the snapshot code before extending the functionality off the top of it.

Yup. See above. I appreciate your input, but the goal of live migration was set in 2016 with a prototype first demonstrated in 2018. How long do you suggest a developer wait without review feedback before moving forward out of tree?

 
There are experimental patches for all these features that were developed by students at UPB. In a lot of cases, there are open reviews that have been waiting on feedback for ages.

In general, most people don't want to review large experimental patches.

Yup. That approach was attempted with the Warm Migration patches. From slide 17 in Elena's presentation:

First review opened in 2021: https://reviews.freebsd.org/D28270
5 reviews from 2022 starting with https://reviews.freebsd.org/D34717 (same feature split in multiple parts)

A similar request was made recently to Gusev Vitaliy WRT the multiple device support patch which he took ownership of. Thanks for adding feedback to that review BTW. We'll see how that pans out ...

https://reviews.freebsd.org/D35590

 
The case is quite plain. I'm not sure what the solution is to this
problem. I'd love to hear feedback from the community about how I've got
this completely wrong and how the course could be corrected. That would
be something.

My perspective is that it would have been better to focus student efforts on completing the snapshot feature. By completing the snapshot feature, I mean getting the code into a state where it's compiled in by default and no longer considered an experimental feature.

I'm not sure what more to say hear regarding the snapshot feature or what might have been done in the past. We need a solution for the present. If you have any comments related to the follow up reviews submitted by UPB, I'm sure they'd love to hear them.

And lastly: I get that FreeBSD is a non paid volunteer project for most. Without the efforts of folks like Peter, Neel, John and others, there would be no bhyve. I'm not saying that they, as project maintainers, should somehow be doing more. We all have limited time to invest, paid work to do and families to feed. I'm asking if there are other developers that might be willing and able to help with reviews? Is there something the FreeBSD foundation can do help out in situations like these?

Thanks,

-Matthew

--------------PosPzAVxthavBLdS0gbQcyBE--