core dumps are HUGE...
John Baldwin
jhb at freebsd.org
Wed Mar 22 15:20:06 UTC 2006
On Wednesday 22 March 2006 09:11, Chuck Swiger wrote:
> Hi, Jason--
>
> First, thanks for your work on the new jemalloc.
>
> Jason Evans wrote:
> > On Mar 21, 2006, at 2:20 PM, John-Mark Gurney wrote:
> [ ... ]
> >> And yes, I'm pretty sure that I have a world from before your reduction
> >> in chunk size... Having a minimum of a 6meg core doesn't sound like a
> >> good idea... It's definately not going to play nice with small systems..
> >
> > I don't think that a 6 MB core file is the big deal you are making it
> > out to be. A 50 MB core file is a pain, mainly because it takes a long
> > time to dump core.
>
> The smallest processes that people run often are likely going to be /bin/sh and
> the typical fodder of shell scripts (grep, awk, and so forth). Under 4.x,
> /bin/sh tended to have VSIZE of 640K and RSS of ~300K; under 5.x and later,
> /bin/sh has VSIZE of 1.6MB and RSS of ~1MB.
>
> Creating a 50MB corefile from a 2MB VSIZE process is unreasonable. Creating a
> 6MB corefile from a 2MB VSIZE process would be reasonable but probably could be
> improved by a factor of two.
>
> On the other hand, the staticly linked "t" program is:
>
> chuck 62964 0.0 0.1 232 128 p0 TX 8:09AM 0:00.03 /tmp/t
>
> 128K RSS, 232 VSIZE. A 6MB corefile is a factor of 25 larger, which strikes me
> as something that could be improved by more than a factor of ten.
>
> What else is being dumped that's not counted by VSIZE, aside from a header and
> maybe some per-thread state, (although aren't the per-thread stacks already part
> of VSIZE)...?
>
> Can jemalloc only create per-CPU arenas only for processes which are themselves
> multithreaded, when it's running on a multi-CPU system? Would that help reduce
> the amount of allocated but unreferenced memory that is involved for the common
> case of /bin/sh and friends?
I think the better path is to provide sparse coredumps. I.e., when dumping a
core, leave the parts of the process map that are mapped but have no backing
store yet (b/c the pages haven't been touched) sparse by not writing to them,
but just seeking past them. This doesn't require complicating the malloc
implementation just for the sake of a core dump on a CF device.
--
John Baldwin <jhb at FreeBSD.org> <>< http://www.FreeBSD.org/~jhb/
"Power Users Use the Power to Serve" = http://www.FreeBSD.org
More information about the freebsd-current
mailing list