questions about VM_KMEM_SIZE_SCALE

[Terry Lambert]

David Schultz wrote:
> On Tue, Jun 24, 2003, Jay Kuri wrote:
> > Does changing this affect memory available to user programs if it's unused
> > by the kernel?
> 
> No, KVA_PAGES affects the memory available to user programs.  (You
> have a 4 GB address space on i386 to split between user programs
> and the kernel.)  Within the kernel's share of this address space,
> memory is split into submaps, such as the mb_map (for the
> network), buffer_map for the filesystem buffer cache, and the
> kmem_map for just about everything else.  These submaps are
> size-limited to prevent any one of them from getting out of hand.
> 
> The vm_kmem_map is sized automatically based on the amount of
> memory you have.  Specifically,
> 
> kmem_map_size = min(max(VM_KMEM_SIZE, Physical memory/VM_KMEM_SIZE_SCALE),
>                     VM_KMEM_SIZE_MAX)
> 
> The default value for VM_KMEM_SIZE_SCALE is 3, and the default
> VM_KMEM_SIZE_MAX is 200MB.

David is exactly right in what he has said.  Some minor ommisions
about the implications of what he has said, though, are:

1)	The intent of doing this is to ensure that, for a given
	amount of physical memory, you don't grab more than
	1/VM_KMEM_SIZE_SCALE * 100 % of it (default: 50%) for
	the kmem_map.

2)	If you *need* a much larger kmem_map, you are limited to
	200K, no matter how much physical memory you have, unless
	you raise VM_KMEM_SIZE_MAX, as well.

3)	If you want to explicitly control the memory size, you can
	set VM_KMEM_SIZE_SCALE very, very large, which will cause
	the second term in the "max()" expression to approach zero,
	and then set VM_KMEM_SIZE_MAX = VM_KMEM_SIZE = <some value>
	to disable the "min()" expression.

4)	The kmem_map is used to allocate kernel structure having to
	do with memory management; if you have a very large amount
	of memory in your system, you should consider increasing
	VM_KMEM_SIZE_MAX, at a minimum, but realize that you are
	competing for KVA with all ather uses of kernel memory, so
	if your KVA space is 2G, and you have 4G of RAM, and your
	VM_KMEM_SIZE_SCALE is 2 and you set VM_KMEM_SIZE_MAX so it
	doesn't clamp the top end, you can run yourself out of KVA
	space.  IMO, it would be good to use something like
	min(Physical Memory, KVA space) in place of Physical Memory
	as the first term of the min() expression.

5)	It's generally useful to set VM_KMEM_SIZE_MAX huge, and
	then use only VM_KMEM_SIZE_SCALE to adjust how much of
	the physical RAM you allow the kmem_map to use (subject to
	the limits in #4, above).

6)	Physical memory allocated to backing KVA resident maps
	like kmem_map don't reduce the amount of virtual memory
	available to user space processes; however, actually using
	the full KVA space worth of physical memory might mean that
	user space processes compete for physical RAM where they
	wouldn't before, and so it may mean swapping.  Memory in
	KVA maps is generally type-stable and can't be reclaimed
	for user process use (i.e.: the kernel does not page, except
	for speciall allocated memory sections, and they are not
	generally used for anything important enough to get a map
	entry).

-- Terry