From nobody Thu May 25 18:40:04 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4QRxgp2khrz4WXgg; Thu, 25 May 2023 18:40:22 +0000 (UTC) (envelope-from gusev.vitaliy@gmail.com) Received: from mail-lf1-x130.google.com (mail-lf1-x130.google.com [IPv6:2a00:1450:4864:20::130]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4QRxgm5q9Tz414f; Thu, 25 May 2023 18:40:20 +0000 (UTC) (envelope-from gusev.vitaliy@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-lf1-x130.google.com with SMTP id 2adb3069b0e04-4f004cc54f4so2956291e87.3; Thu, 25 May 2023 11:40:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1685040017; x=1687632017; h=references:to:cc:in-reply-to:date:subject:mime-version:message-id :from:from:to:cc:subject:date:message-id:reply-to; bh=/ZvE/4ZJL6lRGsKtZoVnqNP57nQpKxBAIEy13pjO8pU=; b=W2arN12tamDFNBgMSDzm+Ttym9+cAt+M6XINwN1XHfxfV780k2icq1jJK+403PpQQZ YvHvGvWruLoFPKHOLELznL/v0t+tC0bKO/HFMy3mG5XZB+soZKJ6gUPFmoC9FuAT2DyJ tt/33P69vR9Qei9LKgAir+DZU/k9Ynh9iPyXRDkhUVEnVI61tYlNtKs3178JKUcAbbIg R2sB6hQFB4CwVX+0OGhtNabKIESDMM1BbWsT2sU+WuxQ50NnJXCccKn4fqyD83OWo2is eN94WYZjpd4vfK3uG1HbYjtbaVqSSC+gdGaktILbJwI8rE/21JphYu6XEK1DQxxTc6lr anHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685040017; x=1687632017; h=references:to:cc:in-reply-to:date:subject:mime-version:message-id :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=/ZvE/4ZJL6lRGsKtZoVnqNP57nQpKxBAIEy13pjO8pU=; b=fD2kgkejtgjJRlMLOnehYuTVemzHPnQJmru8DU8RkfxV01Y+t3rY3LFRJsYkH/Uu6C Kxcn8Wrh2Pkh2MAJSIdav2Tps6npUp/ieo9qNA9EVo3Uyh/XbRLRwwE/KsXyZBe7mQkp CZKKc/x3Fkw4NOCSrBnsFyGcZKohV3E6wvmPWXV85ijaUd45Cl/EJzQc1qUyrflpFEFw u8R8+vtHO1pMY5hfzKEFnTpxRmPY7n1dkBcP7NPLsZG6A4pv9dZm/32rlNTp1xed7Hv4 W0cC+p4nsTuokSA20oEoh0mFbVNaqjF0NNV+jjUAMDlnBtkgi2R/8StGgb0C+pZtZra7 kiuQ== X-Gm-Message-State: AC+VfDxh+e8sdDTdy/N/hnf4vOKyiomeI/FcZZ/ZcczhvWBSiUlsjS0o N2sf8CKFpBjJvQQ2XAM5hDzdG4MDI7fWxw== X-Google-Smtp-Source: ACHHUZ6rCYLeKZS+fTF+f8d/Q7atXAwYPBKoF0pLOw/NgqwRmN8pzJB2BV7pEwprrL+x8IHR+BWUpw== X-Received: by 2002:ac2:5d24:0:b0:4eb:104b:bf61 with SMTP id i4-20020ac25d24000000b004eb104bbf61mr6497924lfb.58.1685040016541; Thu, 25 May 2023 11:40:16 -0700 (PDT) Received: from smtpclient.apple ([78.140.234.98]) by smtp.gmail.com with ESMTPSA id p21-20020ac246d5000000b004f3ba3b948dsm293112lfo.284.2023.05.25.11.40.15 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 25 May 2023 11:40:15 -0700 (PDT) From: Vitaliy Gusev Message-Id: Content-Type: multipart/alternative; boundary="Apple-Mail=_9EC57A81-2637-440C-A2F4-5E1697C8811A" List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.500.231\)) Subject: Re: BHYVE SNAPSHOT image format proposal Date: Thu, 25 May 2023 21:40:04 +0300 In-Reply-To: Cc: Tomek CEDRO , virtualization@freebsd.org, freebsd-hackers@freebsd.org To: Mario Marietto References: <67FDC8A8-86A6-4AE4-85F0-FF7BEF9F2F06@gmail.com> <8FE14143-1AA9-418E-A497-FEFB99BF6B9F@gmail.com> X-Mailer: Apple Mail (2.3731.500.231) X-Rspamd-Queue-Id: 4QRxgm5q9Tz414f X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; TAGGED_FROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N --Apple-Mail=_9EC57A81-2637-440C-A2F4-5E1697C8811A Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On 25 May 2023, at 19:22, Mario Marietto = wrote: >=20 > Vitaliy, >=20 > what happens if I clone your repo as source code on my FreeBSD system. = Can I test your code directly or not ? Anyway,I think that,before doing = this,I need to follow some kind of tutorial,to understand how the = workflow is. Otherwise I will be not able to test and stress it.=20 You should build kernel and tools, install it. Then you can run bhyve, = bhyvectl to deal with suspend/resume. Please follow=20 9.5. Building and Installing a Custom Kernel https://docs.freebsd.org/en/books/handbook/book/#kernelconfig-building Make sure that BHYVE_SNAPSHOT is enabled. Also look at build(7): https://man.freebsd.org/cgi/man.cgi?build(7) >=20 > On Thu, May 25, 2023 at 3:40=E2=80=AFPM Vitaliy Gusev = > wrote: >>=20 >>=20 >>> On 25 May 2023, at 04:30, Tomek CEDRO > wrote: >>>=20 >>> On Wed, May 24, 2023 at 5:11=E2=80=AFPM Vitaliy Gusev wrote: >>>> Protecting requires more efforts and it should be clearly defined: = what is purpose. If >>>> purpose is having checksum with 99.9% reliability, NVLIST HEADER = can be widen >>>> to have =E2=80=9Cchecksum=E2=80=9D key/value for a Section. >>>=20 >>> Well, this could be optional but useful to make sure snapshot did = not >>> break somehow for instance backup medium error or something like >>> that.. even more maybe a way to fix it.. just a design stage idea :- >>=20 >> Yes, new format can have checksum of a Section data if implemented. >>=20 >>>=20 >>>=20 >>>> If purpose is having crypto verification - I believe sha256 program = should be your choice. >>>=20 >>> My question was more specific to availability of that feature >>> (integrity + repair) rather than specific format :-) >>>=20 >>> The use case here is having a virtual machine (it was VirtualBox) = with >>> a bare os installed, plus some common applications, that is = snapshoted >>> at some point in time, then experimented a lot, restored from >>> snapshot, etc. I had a backup of such vm + snapshot backed up that = got >>> broken somehow. It would be nice to know that something is broken, >>> what is broken, maybe a way to fix :-) >>=20 >>=20 >> =E2=80=9CIntegrity" is a very broad term. What checksum algorithm is = fine enough? >> =20 >> For the instance, ZFS has several options for checksum: >>=20 >> = checksum=3Don|off|fletcher2|fletcher4|sha256|noparity|sha512|skein|edonr >> =20 >>=20 >> Having checksum for a filesystem is strongly recommended. However, If = consider image format, >> it doesn=E2=80=99t need to care about consistency in a file itself. = As example (!) - binary files in a system. >> They don=E2=80=99t have checksum integrated, validation is done by = another program - pkg or another. >>=20 >>=20 >>>=20 >>>=20 >>>> Why do you need modify snapshot image ? Could you describe more? Do = you >>>> modify current 3 snapshot files? >>>=20 >>> Analysis that require ram / nvram modification? Not sure if this is >>> already possible, but may come handy for experimenting with uefi and >>> maybe some OS (features) that will not run with unmodified nvram :-P >>=20 >>=20 >> Sorry I don=E2=80=99t get, why do you need to modify snapshot image, = but not directly vmem on the running >> VM? >>=20 >> Another question, checksum and modifying image - two mutual exclusive = things.=20 >>=20 >>>=20 >>>=20 >>>> If you are talking about compatibility of a Image format - it = should be compatible in >>>> both directions, at least for not so big format changes. >>>>=20 >>>> If consider overall snapshot/resume compatibility - I believe = forward compatibility >>>> is not case and target. Indeed, why do you need to resume an image = created by >>>> a higher version of a program? >>>=20 >>> This happens quite often. For instance there is a bug in application >>> and I need to revert to (at least) one step older version. Then I am >>> unable to work on a file that I just saved (or was autosaved for = me). >>> Firefox profile settings let be the first example. KiCAD file format >>> is another example (sometimes I need to switch to a devel build to >>> evade a nasty blocker bug then anyone else that uses a release is >>> blocked for some months including me myself). >>=20 >> Any additional thing has a cost of development, testing and support. = Current >> Implementation doesn=E2=80=99t support compatibility at all. Having = compatibility in both >> directions can be hard. >>=20 >> For example, if some variable is removed in bhyve, backward = compatibility is fine, >> but forward compatibly is not possible unless that removed variable = is being saved >> into a snapshot image just for forward compatibility. And of course, = it should be tested >> and verified as worked. >>=20 >> Do you like that approach? I don=E2=80=99t think so. So I guess only = backward compatibility >> should be supported to make the snapshot code simple and robust. >>=20 >> Thanks, >> Vitaliy Gusev >>=20 >>=20 >=20 >=20 > --=20 > Mario. --Apple-Mail=_9EC57A81-2637-440C-A2F4-5E1697C8811A Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8

On 25 = May 2023, at 19:22, Mario Marietto <marietto2008@gmail.com> = wrote:

Vitaliy,

what happens if I = clone your repo as source code on my FreeBSD system. Can I test your = code directly or not ? Anyway,I think that,before doing this,I need to = follow some kind of tutorial,to understand how the workflow is. = Otherwise I will be not able to test and stress it. =


You = should build kernel and tools, install it. Then you can run bhyve, = bhyvectl to deal with suspend/resume.

Please = follow 

 9.5. Building and Installing a Custom = Kernel



Make sure that = BHYVE_SNAPSHOT is enabled.

Also look at = build(7):




On Thu, May 25, 2023 at 3:40=E2=80=AFPM Vitaliy = Gusev <gusev.vitaliy@gmail.com> = wrote:


On 25 May 2023, at 04:30, Tomek CEDRO <tomek@cedro.info> wrote:

On = Wed, May 24, 2023 at 5:11=E2=80=AFPM Vitaliy Gusev wrote:
Protecting requires more efforts and it should be clearly = defined: what is purpose. If
purpose is having checksum with 99.9% = reliability, NVLIST HEADER can be widen
to have =E2=80=9Cchecksum=E2=80= =9D key/value for a Section.

Well, this could be = optional but useful to make sure snapshot did not
break somehow for = instance backup medium error or something like
that.. even more maybe = a way to fix it.. just a design stage idea = :-

Yes, new format can have checksum of a = Section data if implemented.



If purpose is = having crypto verification - I believe sha256 program should be your = choice.

My question was more specific to = availability of that feature
(integrity + repair) rather than = specific format :-)

The use case here is having a virtual machine = (it was VirtualBox) with
a bare os installed, plus some common = applications, that is snapshoted
at some point in time, then = experimented a lot, restored from
snapshot, etc. I had a backup of = such vm + snapshot backed up that got
broken somehow. It would be = nice to know that something is broken,
what is broken, maybe a way to = fix = :-)


 =E2= =80=9CIntegrity" is a very broad term. What checksum algorithm is fine = enough?
 
For the instance,  ZFS has = several options for checksum:

checksum=3Don|off|fletcher2|fletcher4|sha256|noparity|sha5= 12|skein|edonr

    =   =  


Having = checksum for a filesystem is strongly recommended. However, If consider = image format,
it  doesn=E2=80=99t need to care about = consistency in a file itself. As example (!)  - binary files in a = system.
They don=E2=80=99t have checksum integrated, = validation is done by another program  - pkg or = another.




Why do you = need modify snapshot image ? Could you describe more? Do you
modify = current 3 snapshot files?

Analysis that require ram = / nvram modification? Not sure if this is
already possible, but may = come handy for experimenting with uefi and
maybe some OS (features) = that will not run with unmodified nvram = :-P


Sorry I = don=E2=80=99t get, why do you need to modify snapshot image, but not = directly vmem on the = running
VM?

Another question, = checksum and modifying image - two mutual exclusive = things. 



If you are = talking about compatibility of a Image format - it should be compatible = in
both directions, at least for not so big format changes.

If = consider overall snapshot/resume compatibility - I believe  forward = compatibility
is not case and target. Indeed, why do you need =  to resume an image created by
a higher version of a = program?

This happens quite often. For instance = there is a bug in application
and I need to revert to (at least) one = step older version. Then I am
unable to work on a file that I just = saved (or was autosaved for me).
Firefox profile settings let be the = first example. KiCAD file format
is another example (sometimes I need = to switch to a devel build to
evade a nasty blocker bug then anyone = else that uses a release is
blocked for some months including me = myself).

Any additional = thing has a cost of development, testing and support. = Current
Implementation doesn=E2=80=99t support compatibility = at all. Having compatibility in both
directions can be = hard.

For example, if some variable is removed = in bhyve, backward compatibility is fine,
but forward = compatibly is not possible unless that removed variable is being = saved
into a snapshot image just for forward compatibility. = And of course, it should be tested
and verified as = worked.

Do you like that approach? I don=E2=80=99= t think so. So I guess only backward compatibility
should be = supported to make the snapshot code simple and = robust.

Thanks,
Vitaliy = Gusev




-- =
Mario.

= --Apple-Mail=_9EC57A81-2637-440C-A2F4-5E1697C8811A--