RLIMIT_DATA and malloc(3) use of mmap(2)

[Maxim Dounin]

Hello!

On Tue, Nov 22, 2011 at 02:44:10PM +0200, Kostik Belousov wrote:

> I was reminded about the patch I wrote for Igor Sysoev some time ago.
> The issue the patch tries to handle is that jemalloc uses mmap() instead
> of sbrk() for pages allocation, and thus RLIMIT_DATA limit is no longer
> effective to put the bounds on the process heap. Since reverting to sbrk
> for such purpose is worse then the issue itself, I proposed a solution
> of 'self-restricting malloc'.

Just a little clarification for others: currently, there is no way 
to *safely* limit memory usage of a process while using jemalloc 
with mmap().

The only thing available is RLIMIT_VMEM, but it's not safe as it 
may be reached on stack grow (leaving no possibility for an 
application to handle this).

> The patch adds a flag MAP_DATALIMIT to the flags argument of mmap(2),
> which instructs the system to account the mapping in the RLIMIT_DATA
> resource count. The malloc(3) also gets new option 'L' to enable
> passing MAP_DATALIMIT to mmap() when allocating pages. By default,
> the 'L' option is not activated.
> 
> Now, if user wants to ensure that process heap is restricted by the
> ulimit -d and still use mmap() for jemalloc, he supplies the option
> using any mechanism. The behaviour is voluntaristic, to prevent the
> trashing use RLIMIT_SWAP.
> 
> Do people consider the facility useful ?

Yes, at least some way to safely limit process memory usage is 
certainly needed.

It's a bit sad this isn't enabled by default, but it's probably 
too late for this.  RLIMIT_DATA was (almost) a noop for too long 
and making it work again to limit all memory allocations will 
break POLA.

> Any comments for the patch itself ?
> 
> http://people.freebsd.org/~kib/misc/map_datalimit.1.patch

Patch looks ok for me (though I'm not a VM expert).

Another possible aproach would be to introduce separate anonymous 
(private?) mmap limit, this will allow to do essentially the same 
thing in a bit more consistent (IMHO) manner.

Maxim Dounin