Why process memory starts so high up in virtual space with FreeBSD malloc?

[Dan Nelson]

In the last episode (Dec 01), Yuri said:
> Giorgos Keramidas wrote:
> > The FreeBSD malloc(3) implementation can use either mmap() or
> > sbrk() to obtain memory from the system.  It does not 'waste a high
> > percentage of memory' but it simply maps only high addresses (with
> > an unmapped 'hole' in lower addresses).
> 
> But the hole it leaves with MALLOC_OPTIONS='dM' is way larger than
> the one left by 'Dm' option. Usually malloc will keep allocating
> addresses higher than this initial value and will never come back and
> fill some parts of this gap. Therefore "wasting" this space.

Have you actually verified that space is wasted?  Note that the default
for malloc is "DM", which lets malloc use both break- and mmap-based
memory.  Depending on the libc version, one or the other will be
preferred, but both will be used if necessary.  Also, unless you have
tuned your kernel, you may only be able to allocate 512MB of memory via
break() (the kern.maxdsize tunable lets you adjust this).  Here's what
I get with a simple test program on a month-old 7.1-PRE (before the
preference got switched to mmap):

$ MALLOC_OPTIONS=Dm ./malloctest
Malloced 535822336 bytes. First: 0x8080000, Last: 0x27e80000
$ MALLOC_OPTIONS=dM ./malloctest
Malloced 2542796800 bytes. First: 0x28200000, Last: 0xbfa00000
$ MALLOC_OPTIONS=DM ./malloctest
Malloced 3078619136 bytes. First: 0x8080000, Last: 0xbfa00000
$ ./malloctest
Malloced 3078619136 bytes. First: 0x8080000, Last: 0xbfa00000

So using only break(), I can allocate 511 MB.  Using only mmap(), I can
allocate 2.36 GB.  Using both (the default) I can allocate 2.86 GB.

-- 
	Dan Nelson
	dnelson at allantgroup.com