Mem leak : malloc/free + pthreads = leakage?
dudu at dudu.ro
Sun Mar 6 12:07:26 UTC 2011
On Sun, Mar 6, 2011 at 6:53 AM, Eric Anderson <anderson at ttel.com> wrote:
> On Mar 5, 2011, at 10:44 AM, Deomid Ryabkov wrote:
> > On 03/05/2011 04:02 AM, Eric Anderson wrote:
> >> Hi all,
> >> I have a moderately threaded userland program (all C) I am working on
> (using pthreads, freebsd 8.1 64bit). It seems to leak memory (using
> standard malloc/free) badly.
> > as opposed to what? OpenBSD? Linux? Windows? why do you think your
> problem is specific to FreeBSD (as evidenced by your post to a
> FreeBSD-specific list) or is related to threaded programs?
> OpenSolaris and Mac OS X. I didn't really assume or state it was specific
> to FreeBSD, just that this scenario was on FreeBSD. I happen to do most of
> the development and testing on OS X and FreeBSD, and I've enjoyed the
> FreeBSD community for a very long time.
> >> I am using pcap to capture packets and process them. I have a handful
> of libs statically linked in (pcap is one, the rest don't seem to matter - I
> can remove them and still see the leak).
> >> Does anyone know of issues regarding malloc/free on multithreaded
> userland apps?
> > hell yeah. it goes like this: you malloc() then forget to free() - boom,
> you have a memory leak.
> > you're welcome.
> Thanks, very insightful.
> > sarcasm aside, those questions still remain: why do you think
> os/libraries are the problem and not your code?
> Because I am tracking all malloc and free calls within the application code
> (aside from libraries) and I can account for all malloc'ed memory and freed
> memory in both count and by bytes, yet looking at ps output shows a very
> different story, and if I leave it run long enough, will consume all memory
> and swap in the system and then be killed off. I wrapped malloc/free in a
> function, and record all memory alloc'ed and free'd. The only memory I
> cannot track is memory alloced and freed by libraries I am pulling in (well,
> can't track easily anyway without hacking through all of their source code).
> > you can't post all of it, ok, and we don't want all of it either. can you
> isolate a specific example of where valid usage of a library causes a leak?
> Not really - if I could, I would have fixed it by now. It's a non-trivial
> issue - which is why I am beginning to suspect something more complicated
> than a "oops I forgot to free" kind of error. Plus, I have seen a few
> people elsewhere on various forums/mailing lists with similar issues
> claiming that switching to the Hoard allocator fixed the problem (which
> doesn't seem to be happy with 32bit FreeBSD - tried it). I have also seen
> various comments about pthreads and memory allocators having apparent leaks
> at some threading level, but not sure.
Had there been a memleak in jemalloc, I'm sure more people would have
spotted it by now. How many pcap_t structs do you use in your app? libpcap
is not threadsafe. FWIW I've been running a pcap/threaded application
continuously for the past couple of years and the memory usage has been
Also, put a couple of printf()s before and after pcap_dispatch()/pcap_loop()
(if you use any of them), to make sure they don't block waiting for your
callback to return (on some platforms it doesn't, so you never get to free
memory in outer frames). This might not necessarily be your case, but it's
worth taking a look at.
> freebsd-hackers at freebsd.org mailing list
> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe at freebsd.org"
Good, fast & cheap. Pick any two.
More information about the freebsd-hackers