stable/13, vm page counts do not add up

[Andriy Gapon]

On 07/04/2021 22:54, Mark Johnston wrote:
> On Wed, Apr 07, 2021 at 10:42:57PM +0300, Andriy Gapon wrote:
>>
>> I regularly see that the top's memory line does not add up (and by a lot).
>> That can be seen with vm.stats as well.
>>
>> For example:
>> $ sysctl vm.stats | fgrep count
>> vm.stats.vm.v_cache_count: 0
>> vm.stats.vm.v_user_wire_count: 3231
>> vm.stats.vm.v_laundry_count: 262058
>> vm.stats.vm.v_inactive_count: 3054178
>> vm.stats.vm.v_active_count: 621131
>> vm.stats.vm.v_wire_count: 1871176
>> vm.stats.vm.v_free_count: 187777
>> vm.stats.vm.v_page_count: 8134982
>>
>> $ bc
>>>>> 187777 + 1871176 + 621131 + 3054178 + 262058
>> 5996320
>>>>> 8134982 - 5996320
>> 2138662
>>
>> As you can see, it's not a small number of pages either.
>> Approximately 2 million pages, 8 gigabytes or 25% of the whole memory on this
>> system.
>>
>> This is 47c00a9835926e96, 13.0-STABLE amd64.
>> I do not think that I saw anything like that when I used (much) older FreeBSD.
> 
> One relevant change is that vm_page_wire() no longer removes pages from
> LRU queues, so the count of pages in the queues can include wired pages.
> If the page daemon runs, it will dequeue any wired pages that are
> encountered.

Maybe I misunderstand how that works, but I would expect that the sum of all
counters could be greater than v_page_count at times.  But in my case it's less.

> This was done to reduce queue lock contention, operations like
> sendfile() which transiently wire pages would otherwise trigger two
> queue operations per page.  Now that queue operations are batched this
> might not be as important.
> 
> We could perhaps add a new flavour of vm_page_wire() which is not lazy
> and would be suited for e.g., the buffer cache.  What is the primary
> source of wired pages in this case?

It should be ZFS, I guess.

-- 
Andriy Gapon