sbrk(2) broken

[Robert Watson]

On Fri, 4 Jan 2008, Igor Mozolevsky wrote:

> On 04/01/2008, Robert Watson <rwatson at freebsd.org> wrote:
>
>> To be clear, in the new world order, instead of getting NULL back from 
>> malloc(3), SIGKILL is delivered to large processes.
>
> Huh??? Again, huh???

FreeBSD allows memory overcommit, both overcommit of physical memory resulting 
in paging, and overcommit of swap space.  For the last few years, resource 
limits on the data segment size, previously observed by malloc(), have 
prevented processes from mallocing enough memory individually to exhaust swap 
on 32-bit systems.  This is arguably a bug, because you actually want a single 
process to be able to allocate enough memory to fill its address space, but 
because the data segment size is used to make address space layout decisions 
from the inception of the process, is rather inate to using sbrk().  Jason's 
new malloc uses mmap() of anonymous memory, which isn't affected by the data 
segment limit, and hence, as a feature, isn't limited by the resouce limit. 
This turns out to be awkward if you have a run-away process, as where 
previously it would simply get back an error when it tried to exceed its 
resource limit, now it simply consumes all your swap, which then results in 
overcommit.

My hope was that we could re-introduce a resource limit on malloc'd memory 
without large changes, but that appears to have been more tricky than hoped. 
The goal is not to prevent overcommit, which is invaluable in UNIX systems due 
to the fork() model which pretty much pre-supposes it by design, rather, to 
prevent exhaustion of swap by a single process if not specifically allowed by 
the administrator (in the same way we limit all sorts of other things, like 
open files, mbufs, socket buffer memory, etc).  The right way to do it is to 
provide a specifically configurable process limit on swap use, the same way we 
did for data segment size, only not data segment size, but that was considered 
likely too risky for 7.0.

Robert N M Watson
Computer Laboratory
University of Cambridge