Re: [List] Cannot find out what uses space in ZFS dataset

From: <freebsd_at_vanderzwan.org>
Date: Fri, 19 Sep 2025 10:59:59 UTC

> On 19 Sep 2025, at 12:40, Andrea Venturoli <ml@netfence.it> wrote:
> 
> On 9/19/25 12:08, freebsd@vanderzwan.org wrote:
> 
>> What you could do is to find largest objects in the dataset using zdb:
>> zdb -ddd zroot/ROOT/default 0:-1:A |sort -h -b -k 5 |tail
>> that should show you the largest objects in the dataset.
>> First column is the object id which is also the inode number.
>> You can find more info on the objects using:
>> zdb -ddd zroot/ROOT/default OBJECTID
>> You can try to find the name using :
>> find / -xdev -inum OBJECTID
> 
> Here:
> 
>> # zdb -ddd zroot/ROOT/default 0:-1:A |sort -h -b -k 5 |tail
>>    186544    2   128K   128K  31.3M     512  57.5M  100.00  ZFS plain file
>>      5738    2   128K   128K  33.2M     512  51.9M  100.00  ZFS plain file
>>         0    6   128K    16K  39.6M     512   176M   29.37  DMU dnode
>>    186545    2   128K   128K  48.4M     512  93.5M   99.60  ZFS plain file
>>    186426    2   128K   128K  53.1M     512   101M  100.00  ZFS plain file
>>    186668    3   128K   128K  76.2M     512   178M  100.00  ZFS plain file
>>    186209    3   128K   128K  91.9M     512   253M  100.00  ZFS plain file
>>    186671    3   128K   128K   135M     512   327M  100.00  ZFS plain file
>>    186427    3   128K   128K   140M     512   334M  100.00  ZFS plain file
>>       360    3   128K   128K  58.8G     512  90.0G  100.00  ZFS plain file
>> # zdb -ddd zroot/ROOT/default 360
>> Dataset zroot/ROOT/default [ZPL], ID 89, cr_txg 8, 62.1G, 105870 objects
>>    Object  lvl   iblk   dblk  dsize  dnsize  lsize   %full  type
>>       360    3   128K   128K  58.8G     512  90.0G  100.00  ZFS plain file
>> # find / -xdev -inum 360
>> #
> 
> Does this make any sense to you?
> 
> 


That's a 90GB file taking up 58.8 GB of space. Definitely a candidate for your excessive disk usage.

> 
> Can it be that a process has opened a huge file, which was later deleted, but is still there due to the process being still alive?
> Would du or the abobe find show this?
> Any way to track either the file or the process?
> (Just an hypotesis of course).
> 

A reboot would kill any process and release the space. But it would not solve the riddle.
You could try lsof, I think the NODE column refers to the inode/object id if the filedescriptor is connected to a file on disk (VREG in the TYPE column)..
Run lsof as root and grep for 360.

Cheers,
	Paul