Squid memory leaks in -stable using libc malloc

David Schultz das at FreeBSD.ORG
Sun Sep 7 23:42:56 PDT 2003

On Sun, Sep 07, 2003, Anders Nordby wrote:
> Hi,
> On Sat, Sep 06, 2003 at 08:17:19AM -0700, Mike Harding wrote:
> > Squid uses more memory than you assign to cache_mem, this is
> > documented in the Squid FAQ, section 8.  cache_mem is sort of a
> > 'suggested' value, it's normal for squid to use a lot more memory than
> > cache_mem.
> I know this, and I know the contents of the FAQ. However, there's a huge
> difference between what top reports as memory used (SIZE) and what cache
> manager reports as total allocated KB. In a running process I have just
> now, top says Squid uses 1235 MB of memory, while cache manager reports
> 663 MB allocated (contents of /proc/<pid>/map attached). That's after
> running for around 10 hours. If I use dlmalloc (I do on one system), the
> numbers are 2291 MB according to top and 2221 MB according to cache
> manager after running for 30 hours. Do you see the difference? To me,
> there seems to be a leak, or at least very unfavourable results with
> phkmalloc and Squid.

phkmalloc satisfies allocations smaller than the system page size
from power-of-two buckets, and larger allocations are rounded up
to a multiple of the page size.  While simple, this scheme works
poorly for some applications.  For example, postgresql makes some
allocations that are slightly larger than a power of two due to
its use of inline buffer tags.  If you use malloc(3)'s 'U' option
in combination with ktrace(1), you can get a good idea of exactly
what sizes Squid is trying to allocate.

Note also that there's a difference between heap size and physical
memory usage.  In a demand-paged virtual memory system, you can
allocate just about as much heap space as you want practically for
free; you only pay for the pages you actually touch.  A better
metric for the memory pressure that an application is exerting on
the system is its resident set size---'RES' in top(1).  phkmalloc
by default is not aggressive about returning memory to the system,
but it usually exhibits very good behavior from the point of view
of the VM system.

More information about the freebsd-stable mailing list