Memory problem with latest malloc.c

[Jason Evans]

Joe Marcus Clarke wrote:
> GNOME applications use a Python driven XML parser to generate help
> document translations.  The engine takes the English XML document and
> applies translations to it.  The tool, xml2po, is installed as part of
> textproc/gnome-doc-utils.  I'm currently working on porting GNOME 2.21,
> and one of the Evolution help doc changes triggered a memory problem on
> my test machine.  Basically, with up to and including rev 1.154 of
> malloc.c, I am able to generate the help file with no errors.  With all
> later revs including 1.160, the Python process balloons up to about 512
> MB of memory, then dies.
> 
> The only malloc config I've done is symlink Aj to /etc/malloc.conf.  I'd
> be happy to provide the file an exact command used, but it might be
> easier to let me know if there's any debugging I could provide that
> would help here.

Revision 1.160 of malloc.c uses sbrk instead of mmap by default.  The 
intent is to make data segment resource limits useful.  So, there are 
two possible explanations I can think of for your problems.  The first 
possibility is that there is still a bug in malloc that you're hitting. 
  However, the other possibility is that the python program you're 
running really needs more than 512 MB of memory, and you're hitting the 
resource limit.

It would be really helpful to me if you run your program with 
MALLOC_OPTIONS=dM and monitor memory usage.  These flags cause mmap to 
be used instead of sbrk, and we can find out from that how much memory 
you really need.  If peak memory usage is substantially different when 
using mmap versus sbrk, there's probably a malloc bug.

This latest round of malloc changes hasn't been much fun.  Thanks for 
your help and patience.

Jason