Re: FYI; 14.3: A discord report of Wired Memory growing to 17 GiBytes over something like 60 days; ARC shrinks to, say, 1942 MiBytes

From: Darrin Smith <beldin_at_beldin.org>
Date: Wed, 13 Aug 2025 07:46:51 UTC 
On Tue, 12 Aug 2025 12:57:39 +0300
Konstantin Belousov <kostikbel@gmail.com> wrote:

 
> Start looking at differences in periodic shots of vmstat -z and
> vmstat -m. It would not catch direct page allocators.

Ok, I hope I'm reading these outputs correctlty...
Looking at vmstat -z I am assuming the 'size' column shows the size of
each malloc bucket and the used indicates the number of buckets used?
(A quick look at vmstat.c pointing me to memstat_get_* suggests I'm on
the right track) This results in numbers around the right order of
magnitude to match my memory.

I have noticed with 3 samples over the last 18 hours (in which time it
looks like about 1/2 of my memory is now wired, which seems a little
execessive, especially considering ZFS is only using about 6 1/2G
accoding to top:

Mem: 1568M Active, 12G Inact, 656M Laundry, 36G Wired, 994M Buf, 12G
Free ARC: 6645M Total, 3099M MFU, 2617M MRU, 768K Anon, 49M Header, 877M
	  4995M Compressed, 5803M Uncompressed, 1.16:1 Ratio Swap:
8192M Total, 198M Used, 7993M Free, 2% Inuse

In the middle of this rang I was building about 1000 packages in
poudriere so it's been busy. 

Interestingly the ZFS ARC size has actually dropped since 9 hours ago
when I took the 2nd measurement (was about 15G then) but that was at
the height of the build and suggests the ARC is expiring older stuff
happily.

So assuming the used * size is correct I saw the following big changes
in vmstat -z:

vm_page:

18 hours ago (before build): 18159063040, 25473990656

9 hours ago (during build) : 27994304512, 29363249152
delta                      : +9835241472, +3889258496

recent sample              : 14337658880, 35773743104
delta			   : -13656645632, +6410493952


NAMEI:

18 hours ago:  	  2 267 478 016

9 hours ago : 	 13 991 848 960
delta	    : 	+11 724 370 944

recent sample: 	 24 441 244 672
delta	     :	+10 449 395 712


zfs_znode_cache:

18 hours ago: 370777296	

9 hours ago : 975800816
delta       : +605023520

recent sample: 156404656
delta        : -819396160

VNODE:

18 hours ago: 440384120

9 hours ago : 952734200	
delta	    : +512350080

recent sample: 159528160
delta        : -793206040

Everything else comes out to smaller numbers, so I assume it's probably
not them.

If Im getting the numbers right I'm seeing various caches
expiring after the poudriere build finished. But that NAMEI seems to be
growing quite extensively still, don't know if that's expected or not :)

I will keep watching these, and hopefully get a sample after the
machine has started killing processess.

If any gurus would like .xml dumps of the vmstat -z & -m outputs I have 
them avaiable (xml easier to import into spreadsheet for me), I can
email them or upload them somewhere suitable.

Darrin














-- 

=b