Possible memory leak in the kernel (contigmalloc)
Vanco, Juraj
juraj.vanco at intel.com
Wed Oct 31 17:13:33 UTC 2018
Hi Konstatin,
what we can directly see, is that the number of inactive pages increases and this happens after running single loop of contigmalloc -> contigfree even with the repeating size of allocated blocks.
The only source of inactive page could come from kernel memory management control structures.
Even if such structure becomes once inactive (cached pages), I do not see any reason that kernel cannot reclaim these inactive cached pages when again the contigmalloc asks for the same space of memory.
I think this cycle should not take all the amount of physical memory without being able to reclaim back when it's out of memory:
int array0[10] = {2097152, 1024, 2101248, 1024, 2097152, 2101248, 2101248, 2097152, 2101248, 1024};
int array1[10] = {1024, 2101248, 2097152, 2101248, 2101248, 2097152, 1024, 2101248, 1024, 2097152};
void *mem_blocks[10];
for (i = 0; i < 10; i++)
mem_blocks[i] = contigmalloc(array0[i], TEST_MEM, 0, 0, ~0, PAGE_SIZE, 0);
for (i = 0; i < 10; i++)
contigfree(mem_blocks[i], array0[i], TEST_MEM);
for (i = 0; i < 10; i++)
mem_blocks[i] = contigmalloc(array1[i], TEST_MEM, 0, 0, ~0, PAGE_SIZE, 0);
for (i = 0; i < 10; i++)
contigfree(mem_blocks[i], array1[i], TEST_MEM);
jv
-----Original Message-----
From: owner-freebsd-stable at freebsd.org [mailto:owner-freebsd-stable at freebsd.org] On Behalf Of Konstantin Belousov
Sent: Tuesday, October 30, 2018 6:36 PM
To: Bennett, Ciunas <ciunas.bennett at intel.com>
Cc: freebsd-stable at freebsd.org
Subject: Re: Possible memory leak in the kernel (contigmalloc)
On Tue, Oct 30, 2018 at 11:15:47AM +0000, Bennett, Ciunas wrote:
> Hi,
> The only other activity that is running is the csh script that is inserting and removing the kernel module.
> I am not using the VM for any other purpose but to run this test.
> In my tests the correlation between memory allocation and moving to inactive list can be seen.
> I don't think any other process is creating the inactive memory.
But it is created. More, since anon private memory is freed (not deactivated) on the process exit, something is accumulating the memory.
The other possibility is that the memory is the caching pages from vnodes, but for this buffers must be created and then reclaimed, which would suggest even more activity on the system.
> Thanks.
>
> -----Original Message-----
> From: Konstantin Belousov [mailto:kostikbel at gmail.com]
> Sent: Tuesday, October 30, 2018 10:12 AM
> To: Bennett, Ciunas <ciunas.bennett at intel.com>
> Cc: freebsd-stable at freebsd.org
> Subject: Re: Possible memory leak in the kernel (contigmalloc)
>
> On Tue, Oct 30, 2018 at 09:46:59AM +0000, Bennett, Ciunas wrote:
> > Hi,
> >
> > I was debugging the issue by viewing the free ques "sysctl
> > vm.phys_free" and also using "show page" in ddb. The inactive memory
> > is never being released back into the free que. I thought that when
> > inactive memory reaches a certain threshold that the kernel will
> > start reclaiming and move it to the free list? In my program this is
> > not happening, the program uses free memory (contigmalloc), and then
> > it is put into the inactive que (contiigfree) when the program frees it.
> Contigfree() does not release memory into inactive queue. By definition, inactive pages have valid content, which is not possible for the pages freed by contigfree().
> contigfree()->kmem_free()->kmem_unback()->vm_page_free().
>
>
> > This inactive memory is never released by the kernel, and the
> > inactive que grows until all the memory is in this que. I have
> > attached a xml sheet that shows the memory usage in the system.
> Inactive memory is not freed, it makes no sense as far as there is
> valid content, which is either not recoverable (anon memory or dirty
> file
> pages) or high-cost to restore (clean file pages, need to re-read from disk). Inactive is processed by pagedaemon when system has memory deficit, and either inactive pages are written to swap, or written to their file backing storage.
>
> I suspect that you have other activities on your system going on, which cause creation of the inactive memory and unrecoverable fragmentation.
> Note that contigmalloc() tries to do defragmentation to satisfy requests, but this is not always possible.
>
>
> > Ciunas.
> >
> > -----Original Message-----
> > From: Konstantin Belousov [mailto:kostikbel at gmail.com]
> > Sent: Friday, October 26, 2018 9:13 PM
> > To: Bennett, Ciunas <ciunas.bennett at intel.com>
> > Cc: freebsd-stable at freebsd.org
> > Subject: Re: Possible memory leak in the kernel (contigmalloc)
> >
> > On Wed, Oct 24, 2018 at 04:27:52PM +0000, Bennett, Ciunas wrote:
> > > Hello,
> > >
> > > I have encountered an issue with a kernel application that I have
> > > written, the issue might be caused by a memory leak in the kernel.
> > > The application allocates and deallocates contiguous memory using
> > > contigmalloc() and contigfree(). The application will fail after a
> > > period of time because there is not enough free contiguous memory
> > > left. There could be an issue with the freeing of memory when
> > > using the contigfree() function.
> > >
> >
> > It is unlikely that there is an issue with a leak, but I would be not surprised if your allocation/free pattern would cause fragmentation on free lists that results in contigmalloc(9) failures after.
> >
> > Look at the vmstat -z/vmstat -m output to see uma and malloc stats.
> > More interesting for your case can be the output from
> > sysctl vm.phys_free
> > which provides information about the free queues and order of free pages on them.
> > --------------------------------------------------------------
> > Intel Research and Development Ireland Limited Registered in Ireland
> > Registered Office: Collinstown Industrial Park, Leixlip, County
> > Kildare Registered Number: 308263
> >
> >
> > This e-mail and any attachments may contain confidential material
> > for the sole use of the intended recipient(s). Any review or
> > distribution by others is strictly prohibited. If you are not the
> > intended recipient, please contact the sender and delete all copies.
>
>
> --------------------------------------------------------------
> Intel Research and Development Ireland Limited Registered in Ireland
> Registered Office: Collinstown Industrial Park, Leixlip, County
> Kildare Registered Number: 308263
>
>
> This e-mail and any attachments may contain confidential material for
> the sole use of the intended recipient(s). Any review or distribution
> by others is strictly prohibited. If you are not the intended
> recipient, please contact the sender and delete all copies.
>
_______________________________________________
freebsd-stable at freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscribe at freebsd.org"
--------------------------------------------------------------
Intel Research and Development Ireland Limited
Registered in Ireland
Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
Registered Number: 308263
This e-mail and any attachments may contain confidential material for the sole
use of the intended recipient(s). Any review or distribution by others is
strictly prohibited. If you are not the intended recipient, please contact the
sender and delete all copies.
More information about the freebsd-stable
mailing list