Re: ZFS missing space

On 12/02/2025 10:55, Frank Leonhardt wrote:
> I've noticed space "go missing" on ZFS before, but not conclusively. 
> This time it's happened on a brand new setup I'm doing some testing on.
>
> This is a clean install of 14.2. The machine rebooted (not sure why) 
> while a large dataset was being received (zfs receive...). The dataset 
> isn't there.
>
> There should be about 6Tb in the pool (depending on how you count a Tb).
>
> This is what I'm seeing as varying estimates of free space:
>
> root@zfs2:/home/fjl # zfs list -t all
> NAME              USED  AVAIL  REFER  MOUNTPOINT
> zr               30.6G  5.12T   140K  /zr
> zr/ROOT          5.12G  5.12T   140K  none
> zr/ROOT/default  5.12G  5.12T  5.12G  /
> zr/data           140K  5.12T   140K  /data
> zr/francis       23.3G  5.12T  23.3G  /zr/francis
> zr/home           343K  5.12T   140K  /home
> zr/home/fjl       203K  5.12T   203K  /home/fjl
> zr/tmp            140K  5.12T   140K  /tmp
> zr/usr           2.19G  5.12T   140K  /usr
> zr/usr/ports     1.16G  5.12T  1.16G  /usr/ports
> zr/usr/src       1.03G  5.12T  1.03G  /usr/src
> zr/var           1.11M  5.12T   140K  /var
> zr/var/audit      140K  5.12T   140K  /var/audit
> zr/var/crash      140K  5.12T   140K  /var/crash
> zr/var/log        401K  5.12T   401K  /var/log
> zr/var/mail       180K  5.12T   180K  /var/mail
> zr/var/tmp        140K  5.12T   140K  /var/tmp
> root@zfs2:/home/fjl # zpool iostat
>               capacity     operations     bandwidth
> pool        alloc   free   read  write   read  write
> ----------  -----  -----  -----  -----  -----  -----
> zr          42.8G  7.22T      7      8   417K  1.16M
>
> As this is a "clean" system I can't figure out where the discrepancy 
> could possibly be coming from. This is beyond the slop value.
>
> Any ideas anyone?
>
> (Incidentally, I'm testing failure modes using some flaky hard drives 
> I acquired on eBay - no errors logged prior to reboot).
>
> Thanks, Frank.


Further: Before the system rebooted itself I had IOstat running:

               capacity     operations     bandwidth
pool        alloc   free   read  write   read  write
----------  -----  -----  -----  -----  -----  -----
zr          2.13T  5.14T      0    980      0   142M
   raidz1-0  2.13T  5.14T      0    980      0   142M
     da0p3       -      -      0    254      0  35.3M
     da1p3       -      -      0    351      0  35.6M
     da2p3       -      -      0     80      0  35.5M
     da3p3       -      -      0    293      0  35.6M
----------  -----  -----  -----  -----  -----  -----

As you can see it has 2.13T allocated, which is what I'd have expected a 
few hours into the receive. After the reboot this 2.13Tb allocated has 
disappeared, but not been added back to the free pool!