Virtual memory consumption (both user and kernel) in modern CURRENT

[Jason Evans]

Brian Candler wrote:
> Would it be possible to have some sort of "slow start" algorithm? For
> example, first block allocated is 1MB, next block allocated is 4MB,
> subsequent blocks are 16MB?
> 
> I don't know anything about the internal data structures of jemalloc, but if
> you treat "4MB" as 4 x 1MB and "16MB" as 16 x 1MB, then you'd have
> effectively a pool of 1MB blocks, but apart from the first few, they'd be
> allocated contiguously.

Each chunk is aligned at an address that is a multiple of the chunk 
size.  This makes it possible to use bitmasking to find the beginning of 
  the containing chunk for any allocation in constant time, with no 
locking.  This is critical to the correct, scalable functioning of the 
allocator.

It would be possible to create chunks that are smaller than the standard 
chunk size, but they would still have to be aligned at multiples of the 
standard chunk size.  This would leave trailing space that wouldn't be 
useful to the allocator.  It would in some cases be possible to extend 
small chunks to the standard size, but this would cause some bad 
interactions with the layout policy (large objects are usually carved 
from the end of the chunk).  As such, creating smaller chunks doesn't 
fit very well with the allocator design.

Thanks,
Jason