From nobody Tue Sep 02 18:32:00 2025 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4cGZ8n0Tb0z678Qp for ; Tue, 02 Sep 2025 18:32:09 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4cGZ8m2kq9z3W0C for ; Tue, 02 Sep 2025 18:32:08 +0000 (UTC) (envelope-from mavbsd@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20230601 header.b=KsWZZOZI; dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=freebsd.org (policy=none); spf=pass (mx1.freebsd.org: domain of mavbsd@gmail.com designates 2a00:1450:4864:20::42f as permitted sender) smtp.mailfrom=mavbsd@gmail.com Received: by mail-wr1-x42f.google.com with SMTP id ffacd0b85a97d-3d44d734cabso2087047f8f.3 for ; Tue, 02 Sep 2025 11:32:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1756837922; x=1757442722; darn=freebsd.org; h=content-transfer-encoding:in-reply-to:from:content-language :references:to:subject:user-agent:mime-version:date:message-id :sender:from:to:cc:subject:date:message-id:reply-to; bh=1zepjmsn7hKn86S1oe0tbeMgIq9lJ9a3nkvrLEWeicQ=; b=KsWZZOZIf+/ou/CAHp2n5b0EARSnji51g0KHAK9zHQBhMAj3lbk9KzU7Dh3QJoh/0u 6MytOSgYVNWps3kKcSxW8kc0qkqz67NhLwlxc5w5Y6VbHFnH+BTwfqSHc9I/xk1zENgr xcE6MCtiBenccKehbdvWZvCU+SteQrmlqttLENjtNCGi8pYsjFgFVXb75kTRmJ5RJhpb E5mqJclDAyNdHorH151WYVJCZ8GnVW0T0OXLaqUK2ySW1DhVg6pH1AimTAoPO5+8pv6q ui44SQ6xly+QbioAHOJB5EM29dCryzewFtQtPDwZx61Nk7N+F/suvMsGzfvmknPi94Op 1nRg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1756837922; x=1757442722; h=content-transfer-encoding:in-reply-to:from:content-language :references:to:subject:user-agent:mime-version:date:message-id :sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=1zepjmsn7hKn86S1oe0tbeMgIq9lJ9a3nkvrLEWeicQ=; b=hEA3XyqqmqxVSvUP1eUNS9FCtQXfyXYSuPhhDqDPoGxnHR0jl1GHcebx2w2v2YRGkh UfQFNbj28Ng6l/GfkJT8WbyuE9OjLslmoI6ah/A0M2gj3iydWbecqPGecwNVgbMI1RPo k4CUurU2PRw72xn3IGgZaWskzBBXcmgqKuTQ33hKbKxNaeKun9Iu9/jNHQjjtdNO/UfS qbjfNLXQi1KBgwHksIU52whVSZNA63sf+G560FiHkNJGMuXZWHBqCahIWzxQbG7ZtD7N lbadvo8JsV1NrtAoLmDEjZrruVyeMU+qOPZP0F5UNjvbml2omWthejAc+BpOcXXMlCFs w+pw== X-Forwarded-Encrypted: i=1; AJvYcCU6D5jOboJge2daqzGv4hF0U2meJV1Ed2ITAH93TUEvgZctGR+sg2Z/k07nPhRrUYX/5TgJwaUUS03tWBgFSBc=@freebsd.org X-Gm-Message-State: AOJu0YwOoAPKAW6W7+Ien+HCXYEg60JmwrsRIadmW0mbHryyVX166Prk UJ31Ph87exh8Km28vci6WD3h7ui6EK9PXADPdv6FQ2kWYVzoz13sXdCtnLhzKRsgYUZYEQ== X-Gm-Gg: ASbGncuOq0MgA3sK/9sK3Na9/Vjm/TOosP0BMxw8x5JbmH1qGs+ouM6TuX+6TOiVlov hDQxooVHNb+Q2s2SWhZ7whLb1os5uU5q/0e10pyFPSksKz6T0tJ3Ch1+X4xbU44wwVd7+E7j4/D vD4vtZVCwo03olNHzDYvODSNrgJO6SVZ+HhzMLltcJ6KLXajABNazVc82C3KwQnuSGcfhyyK+Yh PIllHEiRzTfvZ6u807+UkVlh+uvHq4R8k/IKrvzTf4dBaVXaY+fz5zTe2ZxK7kvT4FHxvpvOQTw 04gDJ0V0WS30A6d70e7k2vggwvhHOmEJ9X/hb156ntE9478sYdQZEwVHWaqfzRpW0l1sLNkhCsM +f5A21tYLM51N4x+UPvqYQxO1nBGlPUjIDxT+X9e77UKFndxjPPLEPdiEMJII X-Google-Smtp-Source: AGHT+IExwg7xLGGc28lLosbMhwt0ZFOXhGPHPzexNxVrVBZgjbMgDEkRvTfwXiUaij7g3AAQ8P2ijw== X-Received: by 2002:a05:6000:2689:b0:3d4:7657:3a42 with SMTP id ffacd0b85a97d-3d476574ecemr5840613f8f.3.1756837921803; Tue, 02 Sep 2025 11:32:01 -0700 (PDT) Received: from [192.168.1.3] (89-109-102-5.ptr89.nosmadeira.net. [89.109.102.5]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3d2250115fdsm15120300f8f.40.2025.09.02.11.32.00 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 02 Sep 2025 11:32:01 -0700 (PDT) Message-ID: <9d8f3fa4-b550-4d49-9724-a9eaa08ae7c9@FreeBSD.org> Date: Tue, 2 Sep 2025 14:32:00 -0400 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: reviving ZFS in broken sm_start+sm_size state To: Dmitry Morozovsky , freebsd-current@FreeBSD.org References: Content-Language: en-US From: Alexander Motin In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.10 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FORGED_SENDER(0.30)[mav@FreeBSD.org,mavbsd@gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20230601]; DMARC_POLICY_SOFTFAIL(0.10)[freebsd.org : SPF not aligned (relaxed), DKIM not aligned (relaxed),none]; MIME_GOOD(-0.10)[text/plain]; RCPT_COUNT_TWO(0.00)[2]; FROM_HAS_DN(0.00)[]; MIME_TRACE(0.00)[0:+]; ARC_NA(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; TO_DN_SOME(0.00)[]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; FREEMAIL_ENVFROM(0.00)[gmail.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_NEQ_ENVFROM(0.00)[mav@FreeBSD.org,mavbsd@gmail.com]; DKIM_TRACE(0.00)[gmail.com:+]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; MID_RHS_MATCH_FROM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::42f:from] X-Rspamd-Queue-Id: 4cGZ8m2kq9z3W0C Hi Dmitry, This is a space map corruption, that could happen even some time before the reboot. You should be able to import the pool read-only to evacuate the data, since read-only import does not load space maps. Unfortunately without having any reproduction of the actual corruption we might not be able to understand how it happened. It might be either software of hardware, so unless you have ECC RAM, you may wish to test it. You may also try to use `zdb -emmmm ...` to dump the metaslabs on the pool and look for more corruptions and their patterns, hoping it give any more ideas. On 01.09.2025 11:22, Dmitry Morozovsky wrote: > Dear colleagues, > > after some (AFAIR clean) reboot of current with ZFS-on-root I had (OCRed from > mobile photo but hopefully good enough) unbootable system with the following > panic: > > --- 8< --- > panic: VERIFY3U(entry_offset, <, sm->sm_start + sm->sm_size) failed (1847270282567680 < 92341796864) > > cpuid = 2 > time = 1756738203 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0149c856d0 > vpanic at vpanic+0x136/frame 0xfffffe8149c85800 > spl_panic at spl_panic+0x3a/frame 0xfffffe0149c85860 > space_map_iterate() at space_map_iterate+0x3b1/frame 0xfffffe0149c85920 > space_map_load_length() at space_map_load_length+0x5f/frame 0xfffffe8149c85970 > metaslab_load() at metaslab_load+0x529/frame 0xfffffe8149c85a40 > metaslab_activate() at metaslab_activate+0x46/frame 0xfffffe8149c85a88 > metaslab_alloc_dva_range() at metaslab_alloc_dva_range+0x7f9/frame 0xfffffe0149c85bb0 > metaslab_alloc_range() at metaslab_alloc_range+8x2c2/frame 8xfffffe8149c85c70 > metaslab_alloc() at metaslab_allo > zio_dva_allocate() at 0xfffffe0149c85cc0 > zio_execute() at zio iraframe 0xfffffe0149c85e10/frame 0xfffffe0149c85e40 > taskqueue_run_locked) at taskqueue_run_locked+0x1c2/frame 0xfffffe0149c85ec0 > taskqueue_thread_loop() at taskqueue_thread_loop+0xd3/frame 0xfffffe0149c85ef0 > fork_exit() at fork_exit+0x82/frame 0xfffffe0149c85f30 > fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0149c85f30 > --- trap 0, rip = 0, rsp = 0, rbp = 0 --- > KDB: enter: panic > [ thread pid 0 tid 101011] > Stopped at > --- 8< --- > > attempts to boot from last snapshot and/or trying to boot from PRERELEASE and > zpool import lead to exactly the same results, even with different '-F' > options: > > pool *seems* to be importable but actually isn't due to mad entry_offset as I > can see from source > > any hints how could I resolve this? the pool content itself is not **very** > important, but avoiding recreation would be nice -- Alexander Motin