svn commit: r270759 - in head/sys: cddl/compat/opensolaris/kern cddl/compat/opensolaris/sys cddl/contrib/opensolaris/uts/common/fs/zfs vm

[John Baldwin]

On Wednesday, September 10, 2014 11:29:21 PM Slawa Olhovchenkov wrote:
> On Wed, Sep 10, 2014 at 02:29:30PM -0400, John Baldwin wrote:
> > > Application must change behaviour when reach limit (run GC, switch
> > > algorithm and etc.).
> > > But mmap of big data file for scaning and processing don't touch this
> > > limit.
> > > Must be mmap of some temoprary file touch this limit? I don't know.
> > > Must be MAP_ANON touch this limit? I think yes.
> > > How to make distinction of mmap data file for processing and mmap
> > > temporary file for bypass this limit? I don't know.
> > 
> > Consider also mmap() of shm_open(SHM_ANON).
> 
> No, shm limited separatly. mmap of shm_open(SHM_ANON) don't need to adjust
> this limit.

It would be another trivial way of escaping RLIMIT_DATA however.

> > > PS: question about jemalloc. How I can stop jemalloc to give back
> > > memory to OS for some application?
> > 
> > You would have to hack pages_purge() in chunk_mmap.c to always return true
> > and not call madvise().
> 
> And got memory leak?

No, that doesn't leak memory.  jemalloc() considers that address range free 
regardless and will always reuse it for a future allocation.  This is just a 
hint to let the VM system take back the backing pages because they are 
currently not in use by the application, and if the address is reused, the 
current contents of the pages are not important (can be thrown away).

> I am talk about change behaviour from periodic mmap and madvise memory
> to only mmap when need and leave for future use when freed by
> aplication. I think (may be wrong -- please correct) in some cases
> mmap/madvise will be too often.

The madvise() does not cause a future mmap().  You would have to munmap() to 
require a future mmap().  Instead, the madvise() can result in the page being 
released back to the system.  Later when that address is reused by jemalloc, 
the process would fault on the address requiring the VM to locate a new page.  
Disabling the madvise() only prevents that fault from occurring (However, 
leaving all the pages around and marked as in-use may increase memory pressure 
and cause you to swap and still have to fault the page back in in the future, 
but at greater expense.)

-- 
John Baldwin