[rfc] enabling MALLOC_PRODUCTION on -HEAD for now, until jemalloc has been taught to have some run time selectable debug options

[Jason Evans]

On Jan 20, 2013, at 7:26 PM, Ian Lepore <ian at FreeBSD.org> wrote:
> On Sat, 2013-01-19 at 22:26 -0800, Adrian Chadd wrote:
>> I'd like to enable MALLOC_PRODUCTION on -HEAD.
>> 
>> I'm currently recompiling my libc on this g4 powerbook because the
>> -HEAD snapshots don't have it enabled by default; just to get some
>> damned decent performance out of this thing.
>> 
>> I'll work with Jason and others (eg Ian) who have a vested interest in
>> trying to get it to run better out of the box, but still have the
>> debug options available for people who wish to debug things.
> 
> I've been investigating this today and have some information.
> 
> With MALLOC_PRODUCTION defined there is no problem, even on small
> embedded systems.  Without MALLOC_PRODUCTION we've basically got two
> problems:
> 
>      * Every program has a minimum resident size of about 8MiB, and
>        that's fatal on a small-memory embedded system.
>      * Performance is bad.  This is at least in part due to the expense
>        of faulting in 8MiB of zeroed pages, and that's especially
>        noticible in utilities that should be small and fast.  There
>        could be other causes as well.
> 
> I think I've tracked the cause of the 8MiB resident size to a particular
> sanity check, which validates whether memory that was supposed to have
> been zeroed actually was.  I think this check makes sense in some cases,
> and not in others.  It almost certainly doesn't make sense if the memory
> was freshly obtained from mmap().

Now I remember planning to remove the zeroed memory validation because of its high cost, but deciding otherwise at the last moment.  Literally the day before I released jemalloc 3.0.0, the validation code caught this (in the context of FreeBSD's libc):

	Fix large calloc() zeroing bugs.
	http://www.canonware.com/cgi-bin/gitweb.cgi?p=jemalloc.git;a=commitdiff;h=d8ceef6c5558fdab8f9448376ae065a9e5ffcbdd

Since 3.0.0 though there has been some refactoring that makes it feasible (and clean) to move the validation code from chunk_alloc() to chunk_recycle().  Ian sent me such a patch (better than the one I would have come up with myself) that I will test and integrate upstream.  In the meanwhile I think we should try Ian's patch before resorting to MALLOC_PRODUCTION.

Jason