From nobody Tue May 23 16:05:31 2023 X-Original-To: virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4QQfLL6j8bz4CdWY; Tue, 23 May 2023 16:05:46 +0000 (UTC) (envelope-from gusev.vitaliy@gmail.com) Received: from mail-lj1-x22b.google.com (mail-lj1-x22b.google.com [IPv6:2a00:1450:4864:20::22b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4QQfLK6kZmz4Nwf; Tue, 23 May 2023 16:05:45 +0000 (UTC) (envelope-from gusev.vitaliy@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20221208 header.b=N7L+vi5t; spf=pass (mx1.freebsd.org: domain of gusev.vitaliy@gmail.com designates 2a00:1450:4864:20::22b as permitted sender) smtp.mailfrom=gusev.vitaliy@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-lj1-x22b.google.com with SMTP id 38308e7fff4ca-2af29e51722so239631fa.1; Tue, 23 May 2023 09:05:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1684857943; x=1687449943; h=to:cc:date:message-id:subject:mime-version:from:from:to:cc:subject :date:message-id:reply-to; bh=b1A/pwsrBKPrpnOnXMwIjxXuyJjG5a3N5tKA4NqjFPQ=; b=N7L+vi5taGmxOj5jG7/O1uHMQfELuW/wFxufMcpBkonKduQpO8whXPRE3hn22ObW5k JXkBqNomIQxm2252iSo3XlM8FHNzyZtQIpZ3z4JHvcB6bYlyZraH4QWkbV4pyhw4BwU3 JtQma1/grVgnpkmGWBMSiTaGuWnErU7FlyetBwN57UFpPyBNMISWWWhRD0lEBD0707je ElUw57mig33AKaU5XTPoftZDhjXB/ekfATI/6ZxsBYagBRf+Xy+LpPVDFeq8CdRVmC1a V4o7RqDX+DDkxT8MHECol38JaR2x2XwGjTLk39xioXXfFREBA1HtpoH7FStnGvQ+Ovdc mRhw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684857943; x=1687449943; h=to:cc:date:message-id:subject:mime-version:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=b1A/pwsrBKPrpnOnXMwIjxXuyJjG5a3N5tKA4NqjFPQ=; b=b9oRNixYJqKqQqwyHStWe/GbGm50U09CP7GP6aQvIt6oGYqdfrFoglVVQL2yJ+yezW bQ4kgB636Txp/nWAoAbZzs/r5FIPdMQ6vIue/SAfQ4Jobx0nGXH4OxZjzESyYskjL0CQ ocWsESPp3Tt6gjIM7b+xRy+fqu69AvEkg+9sXr0alrKhBtDMoEN9OWHR/z2UZY3/Zrl9 QKBzezVkelAN0itf2bZKe32HuYgFPmS+96x0L69KxMJfLiBs3VZTJEYUGRXUBLPr7G5D gsSzsJNq0bE4nmM/aEYPyx258MsJMymfpxk0RpSYN7GU5twoai4uAoOIi+9Dgo3Xbiry /HAQ== X-Gm-Message-State: AC+VfDy2oC69NAaN0MITqUach21OR41UbgAxl6fmPhGm0oCtJkW3pa1H zT4cRmxMVrEYpw6fiyhDU7UFiZzaBCA= X-Google-Smtp-Source: ACHHUZ7Ht9U8+nVs76wvqwgPUZr66GEma0XvKqoh13Sa+5eDUOrMjmxnF6bsoG8r1LYeSMmwyoLWRQ== X-Received: by 2002:a2e:a318:0:b0:2a8:a6a9:4303 with SMTP id l24-20020a2ea318000000b002a8a6a94303mr5184581lje.8.1684857942775; Tue, 23 May 2023 09:05:42 -0700 (PDT) Received: from smtpclient.apple ([78.140.234.98]) by smtp.gmail.com with ESMTPSA id l7-20020a2e3e07000000b002af25598f07sm1659025lja.78.2023.05.23.09.05.42 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 23 May 2023 09:05:42 -0700 (PDT) From: Vitaliy Gusev Content-Type: multipart/alternative; boundary="Apple-Mail=_574E1CEA-F86F-46F0-AC0E-FD372008C2F5" List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.500.231\)) Subject: BHYVE SNAPSHOT image format proposal Message-Id: <67FDC8A8-86A6-4AE4-85F0-FF7BEF9F2F06@gmail.com> Date: Tue, 23 May 2023 19:05:31 +0300 Cc: freebsd-hackers@freebsd.org To: virtualization@freebsd.org X-Mailer: Apple Mail (2.3731.500.231) X-Spamd-Result: default: False [-3.44 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.94)[-0.945]; MV_CASE(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20221208]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; BLOCKLISTDE_FAIL(0.00)[2a00:1450:4864:20::22b:server fail,78.140.234.98:server fail]; TAGGED_FROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::22b:from]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_NONE(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; DKIM_TRACE(0.00)[gmail.com:+]; FROM_EQ_ENVFROM(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_TLS_LAST(0.00)[]; MLMMJ_DEST(0.00)[virtualization@freebsd.org,freebsd-hackers@freebsd.org] X-Rspamd-Queue-Id: 4QQfLK6kZmz4Nwf X-Spamd-Bar: --- X-ThisMailContainsUnwantedMimeParts: N --Apple-Mail=_574E1CEA-F86F-46F0-AC0E-FD372008C2F5 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Hi, Here is a proposal for bhyve snapshot/checkpoint image format = improvements. It implies moving snapshot code to nvlist engine.=20 Current snapshot implementation has disadvantages: 3 files per snapshot: .meta, .kern, vram Binary Stream format of data. Adding optional variable - breaks resume Removing variable - breaks resume Changing saved order of variables - breaks resume Hard to get information about what is saved and decode. Hard to debug if somethings goes wrong No versions. If change code, resume of an old images can be passed, but with UB. New nvlist implementation should solve all things above. The first step = - improve snapshot/checkpoint saving format. It eliminates three files = usage per a snapshot. 1. BHYVE SNAPSHOT image format: =20 = +=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2= =80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80= =94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94= =E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2= =80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94+ | HEADER PHYS - 4096 BYTES | = +=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2= =80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80= =94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94= =E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2= =80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94+ | | | DATA | | | = +=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2= =80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80= =94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94= =E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2= =80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94+ 2. HEADER PHYS format:=20 0 = +=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2= =80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80= =94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94= =E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2= =80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94+=20= | IDENT STRING - 64 BYTES | 64 = +=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2= =80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80= =94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94= =E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2= =80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94+ =20= | NVLIST SIZE - 4 BYTES | NVLIST DATA | 72 = +=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2= =80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80= =94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94= =E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2= =80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94+ | | | NVLIST DATA | | | 4096 = +=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2= =80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80= =94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94= =E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2= =80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94+ IDENT STRING - Each producer can set its own value to specify image. NVLIST SIZE - The following packed header nvlist data size. NVLIST DATA - Packed nvlist header data. 4KB should be enough for the HEADER to keep basic information about = Sections. However, it can be enlarged lately, without breaking backward compatibility.=20 3. NVLIST HEADER consists of Sections in the following format: Name - string Type: string: =E2=80=9Ctext, - plain text, =E2=80=9Cnvlist=E2=80=9D - packed nvlist, =E2=80=9Cbinary=E2=80=9D - raw binary data. Size - Size of section - uint64 Offset - Offset in image format - uint64 Predefined sections: =E2=80=9Cconfig=E2=80=9D, =E2=80=9Cdevices=E2=80= =9D, =E2=80=9Ckernel=E2=80=9D, =E2=80=9Cmemory=E2=80=9D.=20 4. EXAMPLE: IDENT STRING: "BHYVE CHECKPOINT IMAGE VERSION 1" NVLIST HEADER:=20 [config] config.offset =3D 0x1000 (4096) config.size =3D 0x1f6 (502) config.type =3D "text" [kernel] kernel.offset =3D 0x11f6 (4598) kernel.size =3D 0x19a7 (6567) kernel.type =3D =E2=80=9Cnvlist" [devices] devices.offset =3D 0x2b9d (11165) devices.size =3D 0x10145ba (16860602) devices.type =3D "nvlist" [memory] memory.offset =3D 0x1200000 (18874368) memory.size =3D 0x3ce00000 (1021313024) memory.type =3D =E2=80=9Cbinary" SECTIONS: [section "config" size 0x1f6 offset 0x1000]: memory.size=3D1024M x86.strictmsr=3Dtrue x86.vmexit_on_hlt=3Dtrue cpus=3D2 acpi_tables=3Dtrue pci.0.0.0.device=3Dhostbridge pci.0.31.0.device=3Dlpc pci.0.4.0.device=3Dvirtio-net pci.0.4.0.backend=3Dtap0 pci.0.7.0.device=3Dfbuf pci.0.7.0.tcp=3D10.42.0.78:5900 pci.0.7.0.w=3D1024 pci.0.7.0.h=3D768 pci.0.5.0.device=3Dahci pci.0.5.0.port.0.type=3Dcd pci.0.5.0.port.0.path=3D/ISO/ubuntu-22.04.1-live-server-amd64.iso lpc.bootrom=3D/usr/local/share/uefi-firmware/BHYVE_UEFI.fd checkpoint.date=3D"Wed Jan 25 23:48:29 2023" name=3Dubuntu22 [section "kernel" size 0x19a7 offset 0x11f6]: [vm] vm.vds_version =3D 0x1 (1) vm.cpu0.data(BINARY): 00 00 00 00 0D 00 00 00 01 00 00 00 00 00 = 00 00 ... size=3D0x28 vm.cpu1.data(BINARY): 00 00 00 00 0D 00 00 00 01 00 00 00 00 00 = 00 00 ... size=3D0x28 vm.checkpoint_tsc =3D 0xe2e0ac6fbe456 (3991273496896598) [hpet] hpet.vds_version =3D 0x1 (1) hpet.data(BINARY): 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = 00 ... size=3D0x118 [vmx] vmx.vds_version =3D 0x1 (1) vmx.cpu_features =3D 0 (0) vmx.cpu0.vmx_data(BINARY): F0 CC 15 B8 FF FF FF FF 40 B4 21 B9 = FF FF FF FF ... size=3D0x288 vmx.cpu1.vmx_data(BINARY): F0 CC 15 B8 FF FF FF FF 00 00 67 41 = D8 9B FF FF ... size=3D0x288 [ioapic] ioapic.vds_version =3D 0x1 (1) ioapic.data(BINARY): 00 00 01 00 00 00 00 00 00 00 00 00 00 00 = 00 00 ... size=3D0x208 [lapic] lapic.vds_version =3D 0x1 (1) lapic.cpu0.data(BINARY): 00 00 00 00 00 00 00 00 00 00 00 00 00 = 00 00 00 ... size=3D0x460 lapic.cpu1.data(BINARY): 00 00 00 00 00 00 00 00 00 00 00 00 00 = 00 00 00 ... size=3D0x460 [atpit] atpit.vds_version =3D 0x1 (1) atpit.data(BINARY): 00 00 00 00 00 00 00 00 54 AD 51 97 0F 0E 00 = 00 ... size=3D0xa0 [atpic] atpic.vds_version =3D 0x1 (1) atpic.data(BINARY): 01 00 00 00 00 00 00 00 00 00 00 00 01 00 00 = 00 ... size=3D0x84 [pmtimer] pmtimer.vds_version =3D 0x1 (1) pmtimer.uptime =3D 0x26fd133e5cc (2679274464716) [rtc] rtc.vds_version =3D 0x1 (1) rtc.data(BINARY): 0A 00 00 00 00 FE FF FF 10 35 13 3D 40 F7 14 = 00 ... size=3D0x98 =E2=80=94 Thanks, Vitaliy Gusev --Apple-Mail=_574E1CEA-F86F-46F0-AC0E-FD372008C2F5 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 Hi,

Here is a proposal for bhyve = snapshot/checkpoint image format = improvements.

It implies moving snapshot code = to nvlist engine. 

Current snapshot = implementation has disadvantages:

  • 3 files per snapshot: .meta, .kern, = vram
  • Binary Stream format of data.
  • Adding  optional = variable - breaks resume
  • Removing variable - breaks = resume
  • Changing saved order of variables - breaks = resume
  • Hard to get information about what is saved and = decode.
  • Hard to debug if somethings goes wrong
  • No = versions. If change code, resume of an old images can be
    passed, but = with UB.

New nvlist implementation = should solve all things above. The first step -
improve = snapshot/checkpoint saving format. It eliminates three files = usage
per a = snapshot.


1. BHYVE = SNAPSHOT image format:  

+=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2= =80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2= =80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80= =94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94= =E2=80=94+
| =      HEADER PHYS - 4096 BYTES         = |
+=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2= =80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80= =94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94= =E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2= =80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94+
|             =                     =       |
|=                DATA   =                 = |
|     =                     =               = |
+=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94= =E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2= =80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80= =94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94= =E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94+


2. HEADER PHYS = format: 

 0   =  +=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80= =94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94= =E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2= =80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80= =94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94= + 
    =   |        IDENT STRING  - 64 BYTES   =       |
 64   = +=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2= =80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80= =94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94= =E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2= =80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94+ =   
      | NVLIST SIZE  - 4 BYTES =   |  NVLIST DATA |
 72   = +=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2= =80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80= =94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94= =E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2= =80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94+
      | =                     =                     = |
      = |               NVLIST DATA   =             |
      |       =                     =               = |
 4096 = +=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2= =80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80= =94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94= =E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2= =80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94+


IDENT STRING - Each producer = can set its own value to specify image.
NVLIST SIZE  - = The following packed header nvlist data size.
NVLIST DATA - = Packed nvlist header data.

4KB should be enough = for the HEADER to keep basic information about Sections. However, it = can
be enlarged lately, without breaking backward = compatibility. 

3. NVLIST = HEADER consists of Sections in the following = format:

  • Name - = string
  • Type:    string:
    • =E2=80=9Ctext, =   - plain text,
    • =E2=80=9Cnvlist=E2=80=9D - packed = nvlist,
    • =E2=80=9Cbinary=E2=80=9D - raw binary = data.
  • Size - Size of section - uint64
  • Offset - = Offset in image format - uint64

    Predefined sections: =  =E2=80=9Cconfig=E2=80=9D, =E2=80=9Cdevices=E2=80=9D, =E2=80=9Ckernel= =E2=80=9D, =E2=80=9Cmemory=E2=80=9D. 


= 4. EXAMPLE:


 IDENT = STRING:

  =      "BHYVE CHECKPOINT IMAGE VERSION = 1"

 NVLIST = HEADER: 

  = [config]

    =     config.offset =3D 0x1000 (4096)

    =     config.size =3D 0x1f6 (502)

    =     config.type =3D "text"

  = [kernel]

    =     kernel.offset =3D 0x11f6 (4598)

    =     kernel.size =3D 0x19a7 (6567)

    =     kernel.type =3D =E2=80=9Cnvlist"

  = [devices]

    =     devices.offset =3D 0x2b9d (11165)

    =     devices.size =3D 0x10145ba (16860602)

    =     devices.type =3D "nvlist"

  = [memory]

    =     memory.offset =3D 0x1200000 (18874368)

    =     memory.size =3D 0x3ce00000 (1021313024)

    =     memory.type =3D =E2=80=9Cbinary"


 SECTIONS:


 [section = "config" size 0x1f6 offset 0x1000]:

memory.size=3D1024M

x86.strictmsr=3Dtrue

x86.vmexit_on_hlt=3Dtrue

cpus=3D2

acpi_tables=3Dtrue

pci.0.0.0.device=3Dhostbridge

pci.0.31.0.device=3Dlpc

pci.0.4.0.device=3Dvirtio-net

pci.0.4.0.backend=3Dtap0

pci.0.7.0.device=3Dfbuf

pci.0.7.0.tcp=3D10.42.0.78:5900

pci.0.7.0.w=3D1024

pci.0.7.0.h=3D768

pci.0.5.0.device=3Dahci

pci.0.5.0.port.0.type=3Dcd

pci.0.5.0.port.0.path=3D/ISO/ubuntu-22.04.1-live-serv= er-amd64.iso

lpc.bootrom=3D/usr/local/share/uefi-firmware/BHYVE_UE= FI.fd

checkpoint.date=3D"Wed Jan 25 23:48:29 = 2023"

name=3Dubuntu22


 [section "kernel" size 0x19a7 offset = 0x11f6]:

  =  [vm]

    =     vm.vds_version =3D 0x1 (1)

    =     vm.cpu0.data(BINARY): 00 00 00 00 0D 00 00 00 01 00 00 00 = 00 00 00 00 ...  size=3D0x28

    =     vm.cpu1.data(BINARY): 00 00 00 00 0D 00 00 00 01 00 00 00 = 00 00 00 00 ...  size=3D0x28

    =     vm.checkpoint_tsc =3D 0xe2e0ac6fbe456 = (3991273496896598)

  =  [hpet]

    =     hpet.vds_version =3D 0x1 (1)

    =     hpet.data(BINARY): 01 00 00 00 00 00 00 00 00 00 00 00 00 = 00 00 00 ...  size=3D0x118

  =  [vmx]

    =     vmx.vds_version =3D 0x1 (1)

    =     vmx.cpu_features =3D 0 (0)

    =     vmx.cpu0.vmx_data(BINARY): F0 CC 15 B8 FF FF FF FF 40 B4 = 21 B9 FF FF FF FF ...  size=3D0x288

    =     vmx.cpu1.vmx_data(BINARY): F0 CC 15 B8 FF FF FF FF 00 00 = 67 41 D8 9B FF FF ...  size=3D0x288

  =  [ioapic]

    =     ioapic.vds_version =3D 0x1 (1)

    =     ioapic.data(BINARY): 00 00 01 00 00 00 00 00 00 00 00 00 = 00 00 00 00 ...  size=3D0x208

  =  [lapic]

    =     lapic.vds_version =3D 0x1 (1)

    =     lapic.cpu0.data(BINARY): 00 00 00 00 00 00 00 00 00 00 00 = 00 00 00 00 00 ...  size=3D0x460

    =     lapic.cpu1.data(BINARY): 00 00 00 00 00 00 00 00 00 00 00 = 00 00 00 00 00 ...  size=3D0x460

  =  [atpit]

    =     atpit.vds_version =3D 0x1 (1)

    =     atpit.data(BINARY): 00 00 00 00 00 00 00 00 54 AD 51 97 0F = 0E 00 00 ...  size=3D0xa0

  =  [atpic]

    =     atpic.vds_version =3D 0x1 (1)

    =     atpic.data(BINARY): 01 00 00 00 00 00 00 00 00 00 00 00 01 = 00 00 00 ...  size=3D0x84

  =  [pmtimer]

    =     pmtimer.vds_version =3D 0x1 (1)

    =     pmtimer.uptime =3D 0x26fd133e5cc = (2679274464716)

  =  [rtc]

    =     rtc.vds_version =3D 0x1 (1)

    =     rtc.data(BINARY): 0A 00 00 00 00 FE FF FF 10 35 13 3D 40 = F7 14 00 ...  = size=3D0x98


=E2=80=94
Thanks,
Vitaliy = Gusev



= --Apple-Mail=_574E1CEA-F86F-46F0-AC0E-FD372008C2F5--