[Bug 256507] Apparent kernel memory leak in 12-STABLE r368820

From: <bugzilla-noreply_at_freebsd.org>
Date: Wed, 09 Jun 2021 18:31:28 UTC

            Bug ID: 256507
           Summary: Apparent kernel memory leak in 12-STABLE r368820
           Product: Base System
           Version: 12.2-STABLE
          Hardware: amd64
                OS: Any
            Status: New
          Severity: Affects Some People
          Priority: ---
         Component: kern
          Assignee: bugs@FreeBSD.org
          Reporter: dave@jetcafe.org

Created attachment 225666
  --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=225666&action=edit
Memory graph

Consider the following output:

 # sysctl vm.stats.vm | grep count
 vm.stats.vm.v_cache_count: 0
 vm.stats.vm.v_user_wire_count: 0
 vm.stats.vm.v_laundry_count: 0
 vm.stats.vm.v_inactive_count: 121191
 vm.stats.vm.v_active_count: 20836
 vm.stats.vm.v_wire_count: 754310
 vm.stats.vm.v_free_count: 254711
 vm.stats.vm.v_page_count: 3993253

It should be pretty clear that these numbers do not add up. There are missing
memory pages. 

I have some detailed statistics of this machine in prometheus. A graph of the
issue is attached. I calculate "lost memory" by simply adding up all the _count
variables except v_page_count, and then subtracting that sum from v_page_count.

You will note that over time, the system gradually loses free memory.
Eventually this machine will start swapping and then exhaust swap space and
hang. This is one example from a machine that is running relatively few
services. It is not running ZFS. However, I observe the same behavior on a few
other machines with disparate services and some of those are running ZFS.

I have spent some time asking on lists and looking at various sysctl values to
try to determine whether I am missing something or not. I was unable to find
anything relevant, and having come to the freebsd-stable list to find two
others experiencing this issue, I'm filing this bug. 

Any data anyone needs, just ask me. I actually use prometheus_sysctl_exporter
(thanks for that btw!). Thanks in advance. :)

You are receiving this mail because:
You are the assignee for the bug.