Re: BHYVE SNAPSHOT image format proposal

On Wed, May 24, 2023 at 5:11 PM Vitaliy Gusev wrote:
> Protecting requires more efforts and it should be clearly defined: what is purpose. If
> purpose is having checksum with 99.9% reliability, NVLIST HEADER can be widen
> to have “checksum” key/value for a Section.

Well, this could be optional but useful to make sure snapshot did not
break somehow for instance backup medium error or something like
that.. even more maybe a way to fix it.. just a design stage idea :-)


> If purpose is having crypto verification - I believe sha256 program should be your choice.

My question was more specific to availability of that feature
(integrity + repair) rather than specific format :-)

The use case here is having a virtual machine (it was VirtualBox) with
a bare os installed, plus some common applications, that is snapshoted
at some point in time, then experimented a lot, restored from
snapshot, etc. I had a backup of such vm + snapshot backed up that got
broken somehow. It would be nice to know that something is broken,
what is broken, maybe a way to fix :-)


> Small is not so perfect. As the first attempt snapshot code is good. But if you want to get
> values related to some specific device, for example, for NIC or HPET, you cannot get it easily. Please
> try :)
>
> Stream doesn’t have flexibility. It is good for well specified  and long long time discussed protocols
> like XDR (NFS), when it has RFC and each position in the stream is described. Example: RFC1813.
>
> New format with NVLIST has flexibility and is fast enough. Note, ZFS uses nvlist for keeping attributes
> and more another things.

Sorry, I was not really aware of that format!! This looks really solid :-)

https://github.com/fudosecurity/nvlist
https://man.freebsd.org/cgi/man.cgi?query=nvlist


> Why do you need modify snapshot image ? Could you describe more? Do you
> modify current 3 snapshot files?

Analysis that require ram / nvram modification? Not sure if this is
already possible, but may come handy for experimenting with uefi and
maybe some OS (features) that will not run with unmodified nvram :-P


> If you are talking about compatibility of a Image format - it should be compatible in
> both directions, at least for not so big format changes.
>
> If consider overall snapshot/resume compatibility - I believe  forward compatibility
> is not case and target. Indeed, why do you need  to resume an image created by
> a higher version of a program?

This happens quite often. For instance there is a bug in application
and I need to revert to (at least) one step older version. Then I am
unable to work on a file that I just saved (or was autosaved for me).
Firefox profile settings let be the first example. KiCAD file format
is another example (sometimes I need to switch to a devel build to
evade a nasty blocker bug then anyone else that uses a release is
blocked for some months including me myself).


> The most important thing - backward compatibility, i.e. when an image is created
> by an older version of a program, but should be resumed on a new one.
>
> This is target and and intention of this improvement.

Thank you, this looks promising :-) Just another design stage idea to
keep the formats forward and backward compatible at least for some
time and/or have an option to skip unknown feature :-)


> Yes, I know about another formats, like JSON or others. NVLIST is the most
> effective and suitable for the current purposes.

Thank you I know that now :-)

I have switched to bhyve recently and for my use I prefer it now over
virtualbox :-)

Thanks again Vitaliy and good luck with the updates :-)

-- 
CeDeROM, SQ7MHZ, http://www.tomek.cedro.info