RLIMIT_DATA and malloc(3) use of mmap(2)
mdounin at mdounin.ru
Tue Nov 22 15:59:04 UTC 2011
On Tue, Nov 22, 2011 at 02:44:10PM +0200, Kostik Belousov wrote:
> I was reminded about the patch I wrote for Igor Sysoev some time ago.
> The issue the patch tries to handle is that jemalloc uses mmap() instead
> of sbrk() for pages allocation, and thus RLIMIT_DATA limit is no longer
> effective to put the bounds on the process heap. Since reverting to sbrk
> for such purpose is worse then the issue itself, I proposed a solution
> of 'self-restricting malloc'.
Just a little clarification for others: currently, there is no way
to *safely* limit memory usage of a process while using jemalloc
The only thing available is RLIMIT_VMEM, but it's not safe as it
may be reached on stack grow (leaving no possibility for an
application to handle this).
> The patch adds a flag MAP_DATALIMIT to the flags argument of mmap(2),
> which instructs the system to account the mapping in the RLIMIT_DATA
> resource count. The malloc(3) also gets new option 'L' to enable
> passing MAP_DATALIMIT to mmap() when allocating pages. By default,
> the 'L' option is not activated.
> Now, if user wants to ensure that process heap is restricted by the
> ulimit -d and still use mmap() for jemalloc, he supplies the option
> using any mechanism. The behaviour is voluntaristic, to prevent the
> trashing use RLIMIT_SWAP.
> Do people consider the facility useful ?
Yes, at least some way to safely limit process memory usage is
It's a bit sad this isn't enabled by default, but it's probably
too late for this. RLIMIT_DATA was (almost) a noop for too long
and making it work again to limit all memory allocations will
> Any comments for the patch itself ?
Patch looks ok for me (though I'm not a VM expert).
Another possible aproach would be to introduce separate anonymous
(private?) mmap limit, this will allow to do essentially the same
thing in a bit more consistent (IMHO) manner.
More information about the freebsd-current