unexpected behaviour of malloc() on 7.0/amd64

[Andrew MacIntyre]

Jason Evans wrote:
> Andrew MacIntyre wrote:
>> In investigating a Python 2.6rc1 regression test failure on FreeBSD
>> 7.0/amd64, as far as I can tell, malloc() does not return NULL when
>> available memory (including swap) is exhausted - the process just gets
>> KILLed.
>>
>> Using ulimit -v to set a virtual memory use limit below the available
>> memory does result in malloc() returning NULL when the limit is hit.
>>
>> The Python regression test concerned does not fail on FreeBSD 7.0/i386,
>> however the C program below exhibits the unexpected behaviour on both
>> 7.0/amd64 and 7.0/i386.  The C program below does behave as
>> expected on FreeBSD 6.3/i386; I cannot currently test its behaviour on
>> FreeBSD 6.3/amd64.
>>
>> I can't see this behaviour documented in the malloc() man page.
> 
>  From malloc(3):
> 
> ===
> 
> IMPLEMENTATION NOTES
>      Traditionally, allocators have used sbrk(2) to obtain memory, which 
> is suboptimal for several reasons, including race conditions, increased 
> fragmentation, and artificial limitations on maximum usable memory.  
> This allocator uses both sbrk(2) and mmap(2) by default, but it can be 
> configured at run time to use only one or the other.  If resource limits 
> are not a primary concern, the preferred configuration is 
> MALLOC_OPTIONS=dM or MALLOC_OPTIONS=DM.  When so configured, the 
> datasize resource limit has little practical effect for typical 
> applications; use MALLOC_OPTIONS=Dm if that is a concern.  Regardless of 
> allocator configuration, the vmemoryuse resource limit can be used to 
> bound the total virtual memory used by a process, as described in 
> limits(1).
> 
> ===
> 
> If you want a custom python binary that does not use mmap, you can 
> define _malloc_options to "d", or just use MALLOC_OPTIONS in the 
> environment.

Thanks for the reply.  The malloc(3) man page for 7.0 doesn't include
this information, and says sbrk() is only used on i386.

 From the 7-stable version of the man page, which appears to match your 
quote above, I infer that these changes will be in 7.1.

However my reading of the updated man page suggests that the malloc
options string should be "Dm" or "m" to turn off mmap() and only use
sbrk() - could you clarify why "d"?

Thanks,
Andrew.

-- 
-------------------------------------------------------------------------
Andrew I MacIntyre                     "These thoughts are mine alone..."
E-mail: andymac at bullseye.apana.org.au  (pref) | Snail: PO Box 370
        andymac at pcug.org.au             (alt) |        Belconnen ACT 2616
Web:    http://www.andymac.org/               |        Australia