kern/152200: Overflow in vmspace_swap_count()

[Andrey Zonov]

>Number:         152200
>Category:       kern
>Synopsis:       Overflow in vmspace_swap_count()
>Confidential:   no
>Severity:       critical
>Priority:       high
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Sat Nov 13 13:20:04 UTC 2010
>Closed-Date:
>Last-Modified:
>Originator:     Andrey Zonov
>Release:        8.1-RELEASE
>Organization:
>Environment:
FreeBSD xxx.xxx.ru 8.1-RELEASE FreeBSD 8.1-RELEASE #0 r212732M: Tue Mar 30 03:14:51 MSD 2010     root at xxx.xxx.ru:/usr/obj/usr/src/sys/GENERIC  amd64
>Description:
When memory exhasted, [pagedaemon] calls src/sys/vm/vm_pageout.c:vm_pageout_oom(), this function calls src/sys/vm/swap_pager.c:vmspace_swap_count() for swap usage calculation of process.

vmspace_swap_count() uses type `int' for address defference and it often leads to `n' variable overflow and system may kill random swapped processes, not only with big memory usage.
>How-To-Repeat:
# swapon /dev/something
# fetch http://zonov.pp.ru/tests/swapped_test.tbz
# tar xyf swapped_test.tbz
# cd swapped_test
# make test

Be carefully, that may hang up your system!
>Fix:


Patch attached with submission follows:

Index: /usr/src/sys/vm/vm_map.h
===================================================================
--- /usr/src/sys/vm/vm_map.h	(revision 212732)
+++ /usr/src/sys/vm/vm_map.h	(working copy)
@@ -373,6 +373,6 @@
     int flags);
 int vm_map_wire(vm_map_t map, vm_offset_t start, vm_offset_t end,
     int flags);
-int vmspace_swap_count (struct vmspace *vmspace);
+long vmspace_swap_count (struct vmspace *vmspace);
 #endif				/* _KERNEL */
 #endif				/* _VM_MAP_ */
Index: /usr/src/sys/vm/swap_pager.c
===================================================================
--- /usr/src/sys/vm/swap_pager.c	(revision 212732)
+++ /usr/src/sys/vm/swap_pager.c	(working copy)
@@ -2412,12 +2412,12 @@
  *	if the VM object has any swap use at all the associated map entries
  *	count for at least 1 swap page.
  */
-int
+long
 vmspace_swap_count(struct vmspace *vmspace)
 {
 	vm_map_t map = &vmspace->vm_map;
 	vm_map_entry_t cur;
-	int count = 0;
+	long count = 0;
 
 	for (cur = map->header.next; cur != &map->header; cur = cur->next) {
 		vm_object_t object;
@@ -2427,7 +2427,7 @@
 			VM_OBJECT_LOCK(object);
 			if (object->type == OBJT_SWAP &&
 			    object->un_pager.swp.swp_bcount != 0) {
-				int n = (cur->end - cur->start) / PAGE_SIZE;
+				vm_offset_t n = (cur->end - cur->start) / PAGE_SIZE;
 
 				count += object->un_pager.swp.swp_bcount *
 				    SWAP_META_PAGES * n / object->size + 1;


>Release-Note:
>Audit-Trail:
>Unformatted: