From nobody Wed Feb 12 14:03:34 2025 X-Original-To: questions@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4YtKmN6PHWz5mqyq for ; Wed, 12 Feb 2025 14:03:48 +0000 (UTC) (envelope-from pprocacci@gmail.com) Received: from mail-ed1-x536.google.com (mail-ed1-x536.google.com [IPv6:2a00:1450:4864:20::536]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4YtKmN0z7Tz3vHS for ; Wed, 12 Feb 2025 14:03:48 +0000 (UTC) (envelope-from pprocacci@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ed1-x536.google.com with SMTP id 4fb4d7f45d1cf-5deb1266031so1208347a12.2 for ; Wed, 12 Feb 2025 06:03:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739369025; x=1739973825; darn=freebsd.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=cJLC1jobGhO7nq62A3YRiEW2Ilc6BWBD4FHlfDlRWRs=; b=PQvq7FfBL0LNBf/hrxLvmPHprsmF+m4jidxEnmno1TJJv6SDjP7a4J1SAqgsFXjowt PHRpf6wSTJx0Uy0juy7Jy+OJijTy0+s4N7LPKQUkLdZ0hpUjbAyt8Tn23DWsLS1JAyna RAdrSIIMEBd3cgaTLO0u4MjAUKxnUp2ZH9DVkKRVlld9/eoULQ8x9l8jRNi1UDBwWc3a CsL4af11F7LH5fKhoZsKlWAwviatstsImsFJlnq98O6HT3Dw0aPfnsvSLG2G3EnXbweK rkMAdLtXm+YWsYGKz1RnMrIse44Q5CjmO2tWw8NWWqQYXk7iUOUPqvwoXg/b5CFblLXa WXzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739369025; x=1739973825; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=cJLC1jobGhO7nq62A3YRiEW2Ilc6BWBD4FHlfDlRWRs=; b=RealGk0oy7N8HoeWeZS9M/4s2TZ/XkpVka8hEDg9eoGxEFQS88q3zG09YcWZDft1om ZXOXp6E8TnIxH8ET7A+0qXhaJMx1A6KtexmZHb4X8AnyLgIwQ59ZlUlcgypTiAJ8771C ctL3WL77Moax/oHl2J94t6b4SIL4nE2SfRy+KvvUSzx5KhdCkGMp7A/RBqOTxnzlpb47 a5iZCNwnQrIwAdO5VPuKd/gU/DkYOTxrI35SWXBWVgjz6sv+0khmtKA10tozyiDDz31k oHVMpZPs1kQgHETEqLNfx5GzXygwvK8+BAuj55IH60j+wNWtxBpe2Gc49dF1aDOmpsh2 8VDA== X-Gm-Message-State: AOJu0Yz+frcl4RDCxkZx2MgyLhomhq8COwOxKYQgHv0UDsWX1jXccuNx CjfHfLeSMKIBmWMXkwhdRGUsnZkO2nFK1+799kOABUVSho/6yUqKNu1TRQj1KOFUrTKPNLhDEPE ug3lhNhftHUu04fIWlhBHv2FPFzRt X-Gm-Gg: ASbGncuBMDArwIvYzICP2WIuCPoayxJyJ4ghCVgg3v53A0SisLO8wDktTdC/7FrXWxf VFdnayw6RjMuO+bZcXrTIJCm0C0pgKZ7IE6FPlHvJuU98zJDwUz8xZuzj/uBQKQKjOnBwSTM= X-Google-Smtp-Source: AGHT+IHiHLBwIXbxONTbYLGFugwXkC/fJVKhSzd7/K3ngObYRSR5q13YfDNaxvf/5Qn7IZgLdEFFVLgc6wsiUEehFik= X-Received: by 2002:a05:6402:5253:b0:5de:3747:cddf with SMTP id 4fb4d7f45d1cf-5deadd7b8e9mr2988104a12.7.1739369024824; Wed, 12 Feb 2025 06:03:44 -0800 (PST) List-Id: User questions List-Archive: https://lists.freebsd.org/archives/freebsd-questions List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: freebsd-questions@freebsd.org Sender: owner-freebsd-questions@FreeBSD.org MIME-Version: 1.0 References: <78dc7b6a-8e87-4f5e-a560-c7b939b0f47f@fjl.co.uk> <24414b4d-5ab1-4d9b-b83a-cb26f2f20761@fjl.co.uk> In-Reply-To: <24414b4d-5ab1-4d9b-b83a-cb26f2f20761@fjl.co.uk> From: Paul Procacci Date: Wed, 12 Feb 2025 09:03:34 -0500 X-Gm-Features: AWEUYZmlh8q-UrliMHsN7fqv6yXntOdCdn4lR8wcaWiajBEYeiby5o3sYOCy2bU Message-ID: Subject: Re: ZFS missing space To: Frank Leonhardt Cc: questions@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 4YtKmN0z7Tz3vHS X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] On Wed, Feb 12, 2025 at 6:06=E2=80=AFAM Frank Leonhardt wrote: > > On 12/02/2025 10:55, Frank Leonhardt wrote: > > I've noticed space "go missing" on ZFS before, but not conclusively. > > This time it's happened on a brand new setup I'm doing some testing on. > > > > This is a clean install of 14.2. The machine rebooted (not sure why) > > while a large dataset was being received (zfs receive...). The dataset > > isn't there. > > > > There should be about 6Tb in the pool (depending on how you count a Tb)= . > > > > This is what I'm seeing as varying estimates of free space: > > > > root@zfs2:/home/fjl # zfs list -t all > > NAME USED AVAIL REFER MOUNTPOINT > > zr 30.6G 5.12T 140K /zr > > zr/ROOT 5.12G 5.12T 140K none > > zr/ROOT/default 5.12G 5.12T 5.12G / > > zr/data 140K 5.12T 140K /data > > zr/francis 23.3G 5.12T 23.3G /zr/francis > > zr/home 343K 5.12T 140K /home > > zr/home/fjl 203K 5.12T 203K /home/fjl > > zr/tmp 140K 5.12T 140K /tmp > > zr/usr 2.19G 5.12T 140K /usr > > zr/usr/ports 1.16G 5.12T 1.16G /usr/ports > > zr/usr/src 1.03G 5.12T 1.03G /usr/src > > zr/var 1.11M 5.12T 140K /var > > zr/var/audit 140K 5.12T 140K /var/audit > > zr/var/crash 140K 5.12T 140K /var/crash > > zr/var/log 401K 5.12T 401K /var/log > > zr/var/mail 180K 5.12T 180K /var/mail > > zr/var/tmp 140K 5.12T 140K /var/tmp > > root@zfs2:/home/fjl # zpool iostat > > capacity operations bandwidth > > pool alloc free read write read write > > ---------- ----- ----- ----- ----- ----- ----- > > zr 42.8G 7.22T 7 8 417K 1.16M > > > > As this is a "clean" system I can't figure out where the discrepancy > > could possibly be coming from. This is beyond the slop value. > > > > Any ideas anyone? > > > > (Incidentally, I'm testing failure modes using some flaky hard drives > > I acquired on eBay - no errors logged prior to reboot). > > > > Thanks, Frank. > > > Further: Before the system rebooted itself I had IOstat running: > > capacity operations bandwidth > pool alloc free read write read write > ---------- ----- ----- ----- ----- ----- ----- > zr 2.13T 5.14T 0 980 0 142M > raidz1-0 2.13T 5.14T 0 980 0 142M > da0p3 - - 0 254 0 35.3M > da1p3 - - 0 351 0 35.6M > da2p3 - - 0 80 0 35.5M > da3p3 - - 0 293 0 35.6M > ---------- ----- ----- ----- ----- ----- ----- > > As you can see it has 2.13T allocated, which is what I'd have expected a > few hours into the receive. After the reboot this 2.13Tb allocated has > disappeared, but not been added back to the free pool! > > > Have you scrub'd? --=20 __________________ :(){ :|:& };: