ten thousand small processes

[Petri Helenius]

Have you looked at the malloc manpage and tuned the cache size;

     <       Reduce the size of the cache by a factor of two.  The default
             cache size is 16 pages.  This option can be specified multiple
             times.

     >       Double the size of the cache by a factor of two.  The default
             cache size is 16 pages.  This option can be specified multiple
             times.

Pete

----- Original Message ----- 
From: "D. J. Bernstein" <djb at cr.yp.to>
To: <freebsd-performance at freebsd.org>
Sent: Saturday, June 21, 2003 8:58 PM
Subject: ten thousand small processes


> FreeBSD 4.8. Test program: malloc(360); malloc(80); malloc(180);
> malloc(16); malloc(440); sleep(10); _exit(0). Compile statically.
> 
> The program ends up with 44KB RSS. Where is all that DRAM going? The
> program also ends up with 168KB VSZ. Where is all that VM going?
> 
> I don't care much about the 3-page text segment. But I do care about the
> 39 extra pages of VM, and the 8 extra pages of DRAM. There's no obstacle
> to having a small program fit into _one_ page per process; two or three
> can be excused, but 39 is absurd. (Yes, I know that Solaris is worse.)
> 
> At least 2 pages appear to be wasted by exit(), because it brings in a
> chunk of stdio, which uses 84 bytes of data and 316 bytes of bss. The
> libc implementors clearly don't care about 316 bytes of memory, so why
> don't they make those 316 bytes static? Why doesn't the compiler
> automatically merge some bss into data when that saves a page? Why can't
> I omit exit(), manually or automatically, when it's unreachable?
> 
> Furthermore, malloc() appears to chew up a whole new page of DRAM for
> each allocation, plus another page---is this counted in VSZ?---for an
> anonymous mmap. Would it really be that difficult to fit 1076 bytes of
> requested memory into the 3000-odd bytes available at the end of bss?
> 
> I sure hope that there's some better explanation for the remaining 32
> pages than ``Well, we decided to allocate 131072 bytes of memory for the
> stack,'' especially when I'm hard-limiting the stack to 4K before exec.
> 
> ---D. J. Bernstein, Associate Professor, Department of Mathematics,
> Statistics, and Computer Science, University of Illinois at Chicago
> _______________________________________________
> freebsd-performance at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-performance
> To unsubscribe, send any mail to "freebsd-performance-unsubscribe at freebsd.org"
>