jemalloc malloc_usable_size()

[Jason Evans]

Andrea Campi wrote:
> First of all, jemalloc makes no effort (neither here nor in free(3))
> to survive being called with non-malloc'ed pointers. If you do that
> and asserts are on, you will get an abort, but if they are not your
> process will likely die an horrible death--but there's no explicit
> guarantee of that. I suppose that's fine (we are talking major WTFs
> after all) in general, although it would be nice to have more control.
> phkmalloc had more limit checking, so bad pointers (text / stack) were
> reliably detected.
> 
> Worse, malloc_usable_size() fails if you pass it a pointer inside a
> malloc'ed region. Again, you get an assert if that's enable, otherwise
> you probably get a random value.
> The reason for this is in arena_salloc, where the passed pointer is
> compared to the calculated page, to determine whether we are dealing
> with a small or large allocation.
> Unfortunately, I don't see an easy fix for this.

For a bogus pointer, you can also get a segmentation fault, because the 
chunk and/or page may not even be mapped.  There are numerous failure 
modes, and protecting against all of them requires very constraining 
data structure approaches.

> Do you have any suggestion, or should I just give up on using
> malloc_usable_size()?

You should not call malloc_usable_size() with bogus arguments if you 
want to avoid undefined behavior.

> As an aside: regardless of my purely educational exercise, what do you
> guys think of putting such checks in the libc itself? The performance
> impact is relatively small, and the benefits in terms of detecing and
> preventing heap buffer overflows huge. That's all the more important
> with jemalloc, which is (relatively) less safe that phkmalloc in fact
> of such attacks.

The performance impact of adding such safety nets is *not* small.  In 
order to make arbitrary bogus realloc()/malloc_usable_size()/free() 
calls "safe", a substantial amount of locking is necessary, and 
conversion from O(1) to O(lg n) algorithms is necessary, since all 
pointers must be validated against an out-of-band database of valid 
pointers.  The performance consequences are especially dire for 
multi-threaded applications.

If you want a debugging malloc implementation, use something other than 
the system malloc.  malloc cannot be made "safe" without substantial 
performance sacrifices.  C's memory model is inherently dangerous, and 
it makes little sense to try to paper over that hard reality in the base 
system libraries.

Jason