kern/162741: [PATCH] vm_kmem_size miscalculated due to int type overflow sometimes

[Bruce Evans]

The following reply was made to PR kern/162741; it has been noted by GNATS.

From: Bruce Evans <brde at optusnet.com.au>
To: Adam McDougall <mcdouga9 at egr.msu.edu>
Cc: freebsd-gnats-submit at FreeBSD.org, freebsd-bugs at FreeBSD.org
Subject: Re: kern/162741: [PATCH] vm_kmem_size miscalculated due to int type
 overflow sometimes
Date: Wed, 23 Nov 2011 03:48:43 +1100 (EST)

 On Mon, 21 Nov 2011, Adam McDougall wrote:
 
 >> Description:
 
 > [Misformatted lines deleted]
 
 >> Fix:
 >
 > Patch attached with submission follows:
 >
 > --- sys/kern/kern_malloc.c.orig	2011-11-21 12:19:25.712591472 -0500
 > +++ sys/kern/kern_malloc.c	2011-11-21 17:25:11.831042640 -0500
 > @@ -704,10 +704,10 @@
 > 	 * Limit kmem virtual size to twice the physical memory.
 > 	 * This allows for kmem map sparseness, but limits the size
 > 	 * to something sane. Be careful to not overflow the 32bit
 > -	 * ints while doing the check.
 > +	 * ints while doing the check or the adjustment.
 > 	 */
 > 	if (((vm_kmem_size / 2) / PAGE_SIZE) > cnt.v_page_count)
 > -		vm_kmem_size = 2 * cnt.v_page_count * PAGE_SIZE;
 > +		vm_kmem_size = 2 * mem_size * PAGE_SIZE;
 >
 > #ifdef DEBUG_MEMGUARD
 > 	tmp = memguard_fudge(vm_kmem_size, vm_kmem_size_max);
 
 cnt.v_page_count should probably be spelled as mem_size in the check too.
 
 The limit is still garbage for 32-bit systems.  32-bit systems can
 easily have 2-4GB of physical memory.  i386 with PAE can have much
 more.  Overflow can't occur in (2 * cnt.v_page_count * PAGE_SIZE)
 since the original vm_kmem_size is limited to 4G-1 by u_long bogusly
 being 32 bits on all supported 32-bit systems.  But the user can
 misconfigure things so that the original vm_kmem_size is only slightly
 less than 4G.  Then there cannot be that much kva.  But when there is
 >= 2G physical, clamping kva to <= 2*physical has no effect.
 
 VM_KMEM_SIZE_MAX or vm.kmem_size would have to be misconfigured for
 vm_kmem_size to be impossibly large.  This means that the above code
 usually has no effect on 32-bit systems.
 
 Bruce