From nobody Sat May 21 06:23:30 2022 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id A73C41B35D05 for ; Sat, 21 May 2022 06:23:44 +0000 (UTC) (envelope-from tsoome@me.com) Received: from ci74p00im-qukt09090302.me.com (ci74p00im-qukt09090302.me.com [17.57.156.21]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4L4tp75lYcz3sXT for ; Sat, 21 May 2022 06:23:43 +0000 (UTC) (envelope-from tsoome@me.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=me.com; s=1a1hai; t=1653114216; bh=N5U2utD5dDsMb/oU6sVH+OCl6Y7YHJZ57RGqUNi/VV0=; h=From:Message-Id:Content-Type:Mime-Version:Subject:Date:To; b=CTVmt0aUD+bzV70Ct1QsCG7erzszJZRXVCj2Dcn4LTfTuhKhkQfmCA7kF0hd/e6A3 TpTt55Vh0rj4MMjvA88mw/HOur0osITMvEhlWq/WSH/Aki4fDRNlnd1o3/G7xK3AEQ khlrMhwJuXOAd0XZNB1R66KuDqiZ0j6usC+yVpXl/VrXpRCOSiwZj8g9xwHhBIP3HM Svv+Mz274mVDy4fF4z2F+p+dPYYuguPFdS8mzlwjMYq5iF8eWTz7FAXjAW93jjBM9b GDxOroWb1Hq5biFqT2Ia5IekgqLGrLeUfKma3+Tp8QJCJCSdWKFuDbqAk+r7irUf0E KMbFTk7/9TQug== Received: from smtpclient.apple (ci77p00im-dlb-asmtp-mailmevip.me.com [17.57.156.26]) by ci74p00im-qukt09090302.me.com (Postfix) with ESMTPSA id BF6FA5BC054D; Sat, 21 May 2022 06:23:34 +0000 (UTC) From: Toomas Soome Message-Id: <3603BC68-C972-4DC6-9D94-6863699D9950@me.com> Content-Type: multipart/alternative; boundary="Apple-Mail=_7F70D6D3-1C2E-427A-B4B0-7A4F9D422BFF" List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.80.82.1.1\)) Subject: Re: Zpool with latest feature com.delpfix:head_errlog can not be booted from. Date: Sat, 21 May 2022 09:23:30 +0300 In-Reply-To: <2c388d289aaa4ab83ee36622e852adca@lerctr.org> Cc: Johan Hendriks , FreeBSD Current To: Larry Rosenman References: <3f367b95-a418-f298-5fe2-972bf2bc1fd4@gmail.com> <2c388d289aaa4ab83ee36622e852adca@lerctr.org> X-Mailer: Apple Mail (2.3696.80.82.1.1) X-Proofpoint-GUID: Ps7sgifCGtaVOqgtPafx--k85WNeec-S X-Proofpoint-ORIG-GUID: Ps7sgifCGtaVOqgtPafx--k85WNeec-S X-Proofpoint-Virus-Version: =?UTF-8?Q?vendor=3Dfsecure_engine=3D1.1.170-22c6f66c430a71ce266a39bfe25bc?= =?UTF-8?Q?2903e8d5c8f:6.0.138,18.0.572,17.0.605.474.0000000_definitions?= =?UTF-8?Q?=3D2020-02-14=5F11:2020-02-14=5F02,2020-02-14=5F11,2020-01-23?= =?UTF-8?Q?=5F02_signatures=3D0?= X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 adultscore=0 clxscore=1011 spamscore=0 mlxscore=0 malwarescore=0 bulkscore=0 suspectscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2205210037 X-Rspamd-Queue-Id: 4L4tp75lYcz3sXT X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=me.com header.s=1a1hai header.b=CTVmt0aU; dmarc=pass (policy=quarantine) header.from=me.com; spf=pass (mx1.freebsd.org: domain of tsoome@me.com designates 17.57.156.21 as permitted sender) smtp.mailfrom=tsoome@me.com X-Spamd-Result: default: False [-3.33 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; FREEMAIL_FROM(0.00)[me.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ip4:17.57.156.0/24]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[me.com:+]; DMARC_POLICY_ALLOW(-0.50)[me.com,quarantine]; NEURAL_HAM_SHORT(-0.73)[-0.726]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[me.com]; ASN(0.00)[asn:714, ipnet:17.57.156.0/24, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[me.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[me.com:s=1a1hai]; FREEFALL_USER(0.00)[tsoome]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-1.00)[-1.000]; TAGGED_RCPT(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; RCVD_IN_DNSWL_LOW(-0.10)[17.57.156.21:from]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current]; FREEMAIL_CC(0.00)[gmail.com,freebsd.org]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[] X-ThisMailContainsUnwantedMimeParts: N --Apple-Mail=_7F70D6D3-1C2E-427A-B4B0-7A4F9D422BFF Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Hi! You can find binaries from boot.tar = root@freebsd:~ # zpool get all NAME PROPERTY VALUE = SOURCE zroot size 29.6G - zroot capacity 68% - zroot altroot - = default zroot health ONLINE - zroot guid 7047501746656921268 - zroot version - = default zroot bootfs zroot/ROOT/default-15 = local zroot delegation on = default zroot autoreplace off = default zroot cachefile - = default zroot failmode wait = default zroot listsnapshots off = default zroot autoexpand on = local zroot dedupratio 1.00x - zroot free 9.36G - zroot allocated 20.3G - zroot readonly off - zroot ashift 0 = default zroot comment - = default zroot expandsize - - zroot freeing 0 - zroot fragmentation 66% - zroot leaked 0 - zroot multihost off = default zroot checkpoint - - zroot load_guid 17322783153073627560 - zroot autotrim off = default zroot compatibility off = default zroot feature@async_destroy enabled = local zroot feature@empty_bpobj active = local zroot feature@lz4_compress active = local zroot feature@multi_vdev_crash_dump enabled = local zroot feature@spacemap_histogram active = local zroot feature@enabled_txg active = local zroot feature@hole_birth active = local zroot feature@extensible_dataset active = local zroot feature@embedded_data active = local zroot feature@bookmarks enabled = local zroot feature@filesystem_limits enabled = local zroot feature@large_blocks enabled = local zroot feature@large_dnode enabled = local zroot feature@sha512 enabled = local zroot feature@skein enabled = local zroot feature@edonr enabled = local zroot feature@userobj_accounting active = local zroot feature@encryption enabled = local zroot feature@project_quota active = local zroot feature@device_removal enabled = local zroot feature@obsolete_counts enabled = local zroot feature@zpool_checkpoint enabled = local zroot feature@spacemap_v2 active = local zroot feature@allocation_classes enabled = local zroot feature@resilver_defer enabled = local zroot feature@bookmark_v2 enabled = local zroot feature@redaction_bookmarks enabled = local zroot feature@redacted_datasets enabled = local zroot feature@bookmark_written enabled = local zroot feature@log_spacemap active = local zroot feature@livelist enabled = local zroot feature@device_rebuild enabled = local zroot feature@zstd_compress enabled = local zroot feature@draid enabled = local zroot feature@zilsaxattr active = local zroot feature@head_errlog active = local root@freebsd:~ #=20 after re.installing boot programs, it does boot, also does work: root@freebsd:~ # /usr/obj/usr/src/amd64.amd64/stand/userboot/test/test = -d /dev/da0 the fix is already pushed. rgds, toomas > On 21. May 2022, at 03:56, Larry Rosenman wrote: >=20 > Can you let me know when a replacement binary is available for EFI? I = have my buildbox/dev system in a non-bootable > state. It's RAIDZ-1 pool, and no place to put another disk. >=20 > Thanks for any help. > (If can email the replacement binary that would be = wonderful). >=20 >=20 > On 05/20/2022 4:47 am, Toomas Soome wrote: >> I=E2=80=99ll see into it. It would be nice to have at least heads up = message >> about such features, or zfs code does have means to block feature >> upgrade on boot pool. >> Rgds, >> Toomas >>> On 20. May 2022, at 11:39, Johan Hendriks = wrote: >>> =EF=BB=BFI did upgrade my FreeBSD Current and with that i updated my = storage pool and my zroot pool. >>> I did add the new gptboot code on the disk. After the reboot i can = not boot anymore. >>> So i did reinstall the os on one disk of the old zroot mirror pool = and did leave the second untouched. >>> Then i can import the pools. >>> If i boot with the latest snapshot ISO = (FreeBSD-14.0-CURRENT-amd64-20220519-716fd348e01-255696-disc1.iso) i see = the following when i boot. >>> BIOS drive A: is fd0 >>> BIOS drive B: is fd1 >>> >>> BIOS drive K: is disk9 >>> ZFS: unsupported feature: com.delpfix:head_errlog >>> ZFS: pool zroot is not supported >>> ZFS: unsupported feature: com.delpfix:head_errlog >>> ZFS: pool storage is not supported >>> BIOS 624kB/2000420kB available memory >>> Then the OS is loaded, if i then go to the shell of the installer = and do a zpool import, ik can import the pool zroot and storage. So this = snapshot has the latest ZFS version with the com.delpfix:head_errlog = feature. So it looks like the bootloader is not able to use the new = feature and thus renders your system unbootable. >>> regards >>> Johan >=20 > --=20 > Larry Rosenman http://www.lerctr.org/~ler > Phone: +1 214-642-9640 E-Mail: ler@lerctr.org > US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106 --Apple-Mail=_7F70D6D3-1C2E-427A-B4B0-7A4F9D422BFF Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 Hi!

You = can find binaries from boot.tar

root@freebsd:~ # zpool get all
NAME =   PROPERTY                 =       VALUE            =               SOURCE
zroot  size             =               29.6G    =                     =   -
zroot  capacity     =                   68%  =                     =       -
zroot  altroot            =             -        =                     =   default
zroot  health       =                   ONLINE =                     =     -
zroot  guid       =                     = 7047501746656921268            = -
zroot  version      =                   -  =                     =         default
zroot  bootfs             =             zroot/ROOT/default-15  =         local
zroot  delegation           =           on           =                   = default
zroot  autoreplace    =                 off    =                     =     default
zroot  cachefile          =             -        =                     =   default
zroot  failmode     =                   wait =                     =       default
zroot  listsnapshots          =         off            =                 = default
zroot  autoexpand     =                 on     =                     =     local
zroot  dedupratio     =                 1.00x  =                     =     -
zroot  free       =                     = 9.36G                  =         -
zroot  allocated          =             20.3G      =                     = -
zroot  readonly     =                   off  =                     =       -
zroot  ashift             =             0        =                     =   default
zroot  comment      =                   -  =                     =         default
zroot  expandsize           =           -          =                     = -
zroot  freeing      =                   0  =                     =         -
zroot  fragmentation          =         66%            =                 = -
zroot  leaked       =                   0  =                     =         -
zroot  multihost          =             off        =                     = default
zroot  checkpoint     =                 -    =                     =       -
zroot  load_guid          =             17322783153073627560   =         -
zroot  autotrim           =             off        =                     = default
zroot  compatibility    =               off      =                     =   default
zroot  feature@async_destroy  =         enabled          =               local
zroot  feature@empty_bpobj        =     active               =           local
zroot  feature@lz4_compress         =   active                 =         local
zroot  feature@multi_vdev_crash_dump  enabled  =                     =   local
zroot  feature@spacemap_histogram =     active               =           local
zroot  feature@enabled_txg        =     active               =           local
zroot  feature@hole_birth         =     active               =           local
zroot  feature@extensible_dataset     active =                     =     local
zroot  feature@embedded_data  =         active           =               local
zroot  feature@bookmarks        =       enabled            =             local
zroot  feature@filesystem_limits      = enabled                  =       local
zroot  feature@large_blocks         =   enabled                =         local
zroot  feature@large_dnode        =     enabled              =           local
zroot  feature@sha512           =       enabled            =             local
zroot  feature@skein          =         enabled          =               local
zroot  feature@edonr          =         enabled          =               local
zroot  feature@userobj_accounting     active =                     =     local
zroot  feature@encryption   =           enabled        =                 = local
zroot  feature@project_quota  =         active           =               local
zroot  feature@device_removal       =   enabled                =         local
zroot  feature@obsolete_counts      =   enabled                =         local
zroot  feature@zpool_checkpoint       = enabled                  =       local
zroot  feature@spacemap_v2        =     active               =           local
zroot  feature@allocation_classes     = enabled                  =       local
zroot  feature@resilver_defer       =   enabled                =         local
zroot  feature@bookmark_v2        =     enabled              =           local
zroot  feature@redaction_bookmarks    = enabled                  =       local
zroot  feature@redacted_datasets      = enabled                  =       local
zroot  feature@bookmark_written       = enabled                  =       local
zroot  feature@log_spacemap         =   active                 =         local
zroot  feature@livelist         =       enabled            =             local
zroot  feature@device_rebuild       =   enabled                =         local
zroot  feature@zstd_compress        =   enabled                =         local
zroot  feature@draid          =         enabled          =               local
zroot  feature@zilsaxattr         =     active               =           local
zroot  feature@head_errlog        =     active               =           local
root@freebsd:~ # 

after re.installing boot programs, it = does boot, also does work:
root@freebsd:~ # = /usr/obj/usr/src/amd64.amd64/stand/userboot/test/test -d = /dev/da0

the = fix is already pushed.

rgds,
toomas

On 21. May 2022, at 03:56, = Larry Rosenman <ler@lerctr.org> wrote:

Can = you let me know when a replacement binary is available for EFI?  I = have my buildbox/dev system in a non-bootable
state. =  It's RAIDZ-1 pool, and no place to put another disk.

Thanks for any help.
(If = <someone> can email the replacement binary that would be = wonderful).


On 05/20/2022 = 4:47 am, Toomas Soome wrote:
I=E2=80=99ll see into it. It would be nice to have at least = heads up message
about such features, or zfs code does = have means to block feature
upgrade on boot pool.
Rgds,
Toomas
On 20. May 2022, at 11:39, Johan Hendriks = <joh.hendriks@gmail.com> wrote:
=EF=BB=BFI = did upgrade my FreeBSD Current and with that i updated my storage pool = and my zroot pool.
I did add the new gptboot code on the = disk. After the reboot i can not boot anymore.
So i did = reinstall the os on one disk of the old zroot mirror pool and did leave = the second untouched.
Then i can import the pools.
If i boot with the latest snapshot ISO = (FreeBSD-14.0-CURRENT-amd64-20220519-716fd348e01-255696-disc1.iso) i see = the following when i boot.
BIOS drive A: is fd0
BIOS drive B: is fd1
<SNAP>
BIOS drive K: is disk9
ZFS: unsupported = feature: com.delpfix:head_errlog
ZFS: pool zroot is not = supported
ZFS: unsupported feature: = com.delpfix:head_errlog
ZFS: pool storage is not = supported
BIOS 624kB/2000420kB available memory
Then the OS is loaded, if i then go to the shell of the = installer and do a zpool import, ik can import the pool zroot and = storage. So this snapshot has the latest ZFS version with the = com.delpfix:head_errlog feature. So it looks like the bootloader is not = able to use the new feature and thus renders your system unbootable.
regards
Johan

--
Larry Rosenman =             &n= bsp;       http://www.lerctr.org/~ler
Phone: +1 = 214-642-9640 =             &n= bsp;   E-Mail:= ler@lerctr.org
US Mail: 5708 Sabbia Dr, Round Rock, = TX 78665-2106

= --Apple-Mail=_7F70D6D3-1C2E-427A-B4B0-7A4F9D422BFF--