malloc does not return null when out of memory
cswiger at mac.com
Thu Jul 24 16:18:12 PDT 2003
> Yes, I thought briefly about something like this.
> Then I thought 'there's a race condition'.
Where? The FreeBSD implementation is wrapped in a THREAD_LOCK()...?
> Then I realised that other processes might not link against this malloc.
> Then I realised the race condition doesn't even matter; processes will
> still be killed, as the kernel doesn't care that you're still in
> malloc() when the overcommitted memory is touched, it just knows you've
> touched it and there's no actual memory there. This will result in far
> more processes being killed. I believe that's a bad thing.
Someone stated that it was a problem that malloc() returned pointers to virtual
address space that had been mapped but not allocated. This patch does not
guarantee that malloc() will return, but, if malloc() does returns a pointer,
using the memory being pointed to will refer to memory that is allocated.
As Barny Wolff said:
> Won't this merely die in malloc, not return 0?
True. This isn't a perfect solution, but given the choice between:
1) malloc(LOTS) returning a pointer, and then sometime later the program dies
with a bus error when using that memory because no more VM is available, or
2) malloc(LOTS) causing an immediate failure in malloc(),
...choice #2 appears to be significantly better.
Figuring out what went wrong from a coredump or backtrace for #2 when the signal
happens in malloc() should be obvious; determining why the program crashed in
the middle of referencing memory in some large buffer is potentially misleading.
Programs which take care to preallocate regions of memory they need before they
start doing a transaction or some other operation that needs to be atomic would
also prefer #2; the patch I proposed could have a beneficial impact on data
integrity for such programs.
People who encounter programs crashing in malloc() are likely going to continue
to complain about malloc() not returning NULL when the system is out of memory.
If malloc() is referencing memory before returning the pointer, means that the
system is going to reserve VM resources with temporal locality towards memory
_allocation_ rather than memory _reference_. Having the program crash at memory
allocation time rather than usage helps identify when and where this problem
actually happens more clearly, if only by a little bit.
I'm not sure whether allocating memory sooner that way will make it more likely
that brk()/sbrk() or mmap() will return ENOMEM to the libc malloc()
implementation, but if it does not help, perhaps that means something and we've
identified the location of problem more precisely.
More information about the freebsd-stable