From nobody Thu Feb 24 16:35:22 2022 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id CF93E19E02C1 for ; Thu, 24 Feb 2022 16:35:24 +0000 (UTC) (envelope-from ler@lerctr.org) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [192.147.25.65]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "*.lerctr.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4K4JRc0yMWz3Hs6; Thu, 24 Feb 2022 16:35:24 +0000 (UTC) (envelope-from ler@lerctr.org) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=ler2019; h=Content-Transfer-Encoding:Content-Type:Message-ID:References: In-Reply-To:Subject:Cc:To:From:Date:MIME-Version:Sender:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=XPaXFjLfo7QNY0csPMQL9Sqwji4sfxnZdurElldPc0c=; b=xGVYwkyfzqwmMk64lGQVDC90Ys EdAwVp/PZawxvQDacYUSkugAl9F7o5NrsKZeSbOvR4it7riR5Ydnau/GsS60+eCSu1xfEjaGXQSCO peBr1AyisB9xTksWV8gsHAXN4GmbMdkCZ2PEhVIbxsc4MphTFpdpYy9U3A4Hk5lz5+oekxx8rOdng Joa4YlLZrL1iuargW3CGX5s3oJ0+bvG4Y76H34dSk7IIeMOUL1EiR9krIAhAPzTUjELrO05KT3m+v dI+73onjULs1Lx0daAwW6jWdDTCQOJqHdcmfGAYJRxtNuejCHZMxDqiViMPUUI3grjgSFSZMZBC2k +Q15s7DQ==; Received-SPF: pass (thebighonker.lerctr.org: domain of lerctr.org designates 2001:470:1f0f:3ad:bb:dcff:fe50:d900 as permitted sender) client-ip=2001:470:1f0f:3ad:bb:dcff:fe50:d900; envelope-from=ler@lerctr.org; helo=webmail.lerctr.org; Received: from thebighonker.lerctr.org ([2001:470:1f0f:3ad:bb:dcff:fe50:d900]:55722 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLS1.3) tls TLS_AES_256_GCM_SHA384 (Exim 4.95 (FreeBSD)) (envelope-from ) id 1nNH5O-0009EK-Ic; Thu, 24 Feb 2022 10:35:22 -0600 Received: from 2600:1700:210:b18f:cc53:f3ee:b613:c3a6 by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Thu, 24 Feb 2022 10:35:22 -0600 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Date: Thu, 24 Feb 2022 10:35:22 -0600 From: Larry Rosenman To: Alexander Motin Cc: Freebsd current Subject: Re: ZFS PANIC: HELP. In-Reply-To: References: <07c0c9c34b4a4133acab597c96867d27@lerctr.org> <95c7c326-e2f9-6e66-7b97-b9fb2671f4ad@FreeBSD.org> <1b6b2017ba69e6fda1ca237c3016ac61@lerctr.org> <6182c57bf1859482e72af78037c399e4@lerctr.org> <7abcbe88-446f-44b9-c3a7-0997d3430b57@FreeBSD.org> <2bc12c9ee3e6dd71b65079116ff2b845@lerctr.org> <5930f3d2b71c0932f903bb5b88a3c87d@lerctr.org> Message-ID: <5ee0a5ae0d0bc6a365f86e6002ada07c@lerctr.org> X-Sender: ler@lerctr.org Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4K4JRc0yMWz3Hs6 X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=lerctr.org header.s=ler2019 header.b=xGVYwkyf; dmarc=pass (policy=none) header.from=lerctr.org; spf=pass (mx1.freebsd.org: domain of ler@lerctr.org designates 192.147.25.65 as permitted sender) smtp.mailfrom=ler@lerctr.org X-Spamd-Result: default: False [-1.65 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[lerctr.org:s=ler2019]; FREEFALL_USER(0.00)[ler]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; MIME_GOOD(-0.10)[text/plain]; NEURAL_SPAM_SHORT(0.15)[0.151]; NEURAL_HAM_LONG(-1.00)[-1.000]; RCVD_COUNT_THREE(0.00)[3]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[lerctr.org:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[lerctr.org,none]; SUBJ_ALL_CAPS(1.20)[16]; MLMMJ_DEST(0.00)[freebsd-current]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:8166, ipnet:192.147.25.0/24, country:US]; MID_RHS_MATCH_FROM(0.00)[] X-ThisMailContainsUnwantedMimeParts: N On 02/24/2022 10:29 am, Alexander Motin wrote: > On 24.02.2022 10:57, Larry Rosenman wrote: >> On 02/23/2022 9:27 pm, Larry Rosenman wrote: >>> It crashes just after root mount (this is the boot pool and only pool >>> on the system), >>> seeL >>> https://www.lerctr.org/~ler/14-BOOT-Crash.png >> >> Where do I go from here? > > I see 2 ways: 1) Since it is only an assertion and 13 is working (so > far), you may just build 14 kernel without INVARIANTS option and later > recreate the pool when you have time. 2) You may treat it as metadata > corruption: import pool read-only and evacuate the data. If you have > recent enough snapshots you may be able to easily replicate the pool > with all the settings to some other disk. ZIL is not replicated, so > corruptions there should not be a problem. If there are no snapshots, > then either copy on file level, or you may be able to create snapshot > for replication in 13 (on 14 without INVARIANTS), importing pool > read-write. Ugh. The box is a 6 disk R710, and all 6 disks are in the pool. I do have a FreeNAS box with enough space to copy the data out. There ARE snaps of MOST filesystems that are taken regularly. The 13 I'm booting from is the 13 memstick image. There are ~70 filesystems (IIRC) with poudriere, ports, et al. I'm not sure how to build the 14 kernel from the 13 booted box. Ideas? Methods? -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: ler@lerctr.org US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106