Observations on virtual memory operations

doug at safeport.com doug at safeport.com
Tue Dec 29 17:49:51 UTC 2020

On Tue, 29 Dec 2020, Michael Schuster wrote:

> On Tue, Dec 29, 2020, 00:37 Pete Wright <pete at nomadlogic.org> wrote:
>> On 12/28/20 3:25 PM, doug wrote:
>>> I have two servers running jails that "routinely" run out of swapspace
>>> with
>>> no demand paging activity. To try and get a handle on VM/swapspace
>>> management I have been tracking swapinfo vs memory use as measured by
>>> top.
>>> The numbers do not exactly add up but I assume that is not involved in my
>>> issue.
>> <snip>
>>> The other day I caught the system at 73% swapspace used. At this level
>>> the
>>> system was in a near thrashing state in that typing a key got it
>>> echoed in
>>> 10 <--> 30 seconds. There was about 600MB of swapspace at this point. I
>>> would think there is no way to debug this except as a thought experiment.
>> The first thing that comes to mind is do you have the ability to hook
>> any metrics/monitoring onto this system.  For example, I use collectd on
>> my systems to report overall CPU/memory metrics as well as per-process
>> memory metrics.
>> Alternatively you could write a simple shell script that run's "ps" and
>> parses the output of memory utilization on a per-process basis.
>> either of the above approaches should give you some insight into where
>> the memory leak is coming from (assuming you already do not know).
>> one trick i use is to invoke a process with "limits" to ensure it does
>> not exceed a certain amount of memory that I allocate to it. for example
>> with firefox i do this:
>> $ limits -m 6g -v 6g /usr/local/bin/firefox
>> that should at least buy you enough time to investigate why the process
>> needs so much memory and see what you can do about it.
Thank you all for the information and thoughts. If vmstat produces correct 
infomation there is no demand paging. The limiting condition on these 
systems is swapfile space rather than real memory. There are 69 sysctl 
elements dealing with paging and swapfile. If there is documentation (other 
than in C) on these that would be helpful perhaps. Most are totals, demand 
paging rates may be in this set, but not so as I can tell.

The one time I caught the system dying the limiting resource was 
swapspace. There was no paging (last vmstat) and about 670MB left in the 
swapfile. In this state I could recover by restarting apache.

More information about the freebsd-questions mailing list