Re: Cannot find out what uses space in ZFS dataset

On 24/09/2025 09:37, Andrea Venturoli wrote:
> On 9/23/25 16:14, Andrea Venturoli wrote:
>> On 9/21/25 23:21, David Christensen wrote:
>>
>>> Have you done a scrub recently?
>>
>> The pool was last scubbed by periodic a month ago.
>> I've launched a new scrub manually now, but I guess it won't do much, 
>> given the problem arised long ago.
>
> As expected, scrub recovered no space.
>
>
>
>
>> I already deleted those snapshots, remember?
>>
>> I wonder if deleting all the others would solve...
>
> Deleting *all* snapshots didn't help either.
>
>

I haven't followed this up since my initial reply as Paul was doing a 
good job.

I have also posted this reply several times in the last few days but for 
some reason it's not appearing on the list!

Could you post the output of this again, now you've deleted the snapshots?

zfs list -t all -o 
name,mountpoint,canmount,refer,used,usedbychildren,usedbydataset,usedbyrefreservation,usedbysnapshots

My guess is there's confusion about where data in directories is stored. 
It's tempting to think that /usr/bin is stored in pool/usr but this is 
not necessarily the case. Just because a dataset has a mountpoint it 
doesn't not mean it's mounted there, even if child datasets are mounted 
below it. If canmount isn't set to "yes" then the data apparently stored 
in a dataset is stored in it's parent.

As I said in my initial reply, ignore du (or even du -A). There are just 
too many reasons for them to report different usage. And the way zfs 
does deletions it won't show up immediately due to the asynchronous 
garbage collection.

It's possible (but I think unlikely in this case) that space is being 
used by metadata. This normally happens if there's a lot of 
fragmentation (try zpool list -v). Each small fragmented block is going 
to need metadata and possibly alignment padding.

You mentioned you were wanting to do a zfs send. Have you tried "zfs 
send zroot/ROOT@snap | wc -c"

Sparse files can also cause anomalies, but I guess you'd know about 
them. The metadata can get quite large if they're referenced in snapshots.

I have encountered leaks in zpools and they're tricky to find - I've 
sometimes given up where I've suspected faulty hardware.

Final thing before I look at output - you don't have a reservation on 
anything, do you?

Now if someone could tell you why my replies don't make it on to the 
questions mailing list on random day's I'd be really interested! I might 
stick to blog posts in future.

Regards, Frank.