amd64: change VM_KMEM_SIZE_SCALE to 1?

[Peter Wemm]

On Mon, Jul 26, 2010 at 11:19 AM, Andriy Gapon <avg at freebsd.org> wrote:
> on 26/07/2010 20:04 Matthew Fleming said the following:
>> On Mon, Jul 26, 2010 at 9:07 AM, Andriy Gapon <avg at freebsd.org> wrote:
>>> Anyone knows any reason why VM_KMEM_SIZE_SCALE on amd64 should not be set to 1?
>>> I mean things potentially breaking, or some unpleasant surprise for an
>>> administrator/user...

The amd64 kernel has a fixed size limit of kva, of which the kmem_map
must fit inside.  Most consumers of malloc() use the free direct map
region, but there are some notable abusers of malloc (zfs being the
prime offender) that prevent the use of the free direct map region for
its allocations.

I'm not familiar with how VM_KMEM_SIZE_SCALE's calculations work but I
think it would be a crying shame to waste a huge chunk of finite kva
space on systems that aren't handicapped by ZFS's abuse of malloc().
We've run out of kva space on amd64 in the past.

To recap.. the amd64 kernel has a place to do temporary mappings.
This space is finite. 6G on newer systems, 2G on older ones.  This is
most often used to remap discontiguous pages into virtually contiguous
address space.  The kernel also sets up a 1:1 virtual<->physical map
region so it can get to any page on the system without requiring a kva
mapping.

If its clear that changing VM_KMEM_SIZE_SCALE makes sense for the
common case then that's different.

Of course, with machines with 128G / 256G of physical ram either
already here or just around the corner, its time to start thinking
hard about physical ram based scaling calculations again.

That hard limit of 512G of physical ram doesn't seem so distant anymore..
-- 
Peter Wemm - peter at wemm.org; peter at FreeBSD.org; peter at yahoo-inc.com; KI6FJV
"All of this is for nothing if we don't go to the stars" - JMS/B5
"If Java had true garbage collection, most programs would delete
themselves upon execution." -- Robert Sewell