adjust kmem_size for large vm_kmem_size

[Andriy Gapon]

We use kmem_size() function in ZFS ARC sizing code to determine values like
arc_c_min and arc_c_mac.  kmem_size() currently returns vm_kmem_size.
OpenSolaris uses 'physmem' value for the same purpose.

I guess that we use vm_kmem_size because typically it is (much) smaller than
amount of physical memory.
On the other hand, it is possible to set vm_kmem_size to value larger than
physical memory (up to 2x is allowed).  Sometimes it is even advisable to do so
to work around kernel KVA space fragmentation.

I think that we should use smaller of the two parameters when deciding ARC size.
The patch below tries to implement that logic.

What do you think?

--- a/sys/cddl/compat/opensolaris/kern/opensolaris_kmem.c
+++ b/sys/cddl/compat/opensolaris/kern/opensolaris_kmem.c
@@ -115,8 +115,14 @@ zfs_kmem_free(void *buf, size_t size __unused)
 uint64_t
 kmem_size(void)
 {
+	static uint64_t kmem_size;

-	return ((uint64_t)vm_kmem_size);
+	if (kmem_size == 0) {
+		kmem_size = (uint64_t)cnt.v_page_count * PAGE_SIZE;
+		if (kmem_size > vm_kmem_size)
+			kmem_size = vm_kmem_size;
+	}
+	return (kmem_size);
 }

 uint64_t

Perhaps, this should be split into two functions: one that sets kmem_size at
appropriate moment and one that simply returns its value.

-- 
Andriy Gapon