Performance of SheevaPlug on 8-stable

[Mark Tinguely]

>  In message: <4B951CE2.6040507 at semihalf.com>
>              Grzegorz Bernacki <gjb at semihalf.com> writes:
>  : This is probably caused by mechanism which turns of cache for shared
>  : pages.
>  : When I add applied following path:
>  : 
>  : diff --git a/sys/arm/arm/pmap.c b/sys/arm/arm/pmap.c
>  : index 390dc3c..d17c0cc 100644
>  : --- a/sys/arm/arm/pmap.c
>  : +++ b/sys/arm/arm/pmap.c
>  : @@ -1401,6 +1401,8 @@ pmap_fix_cache(struct vm_page *pg, pmap_t pm,
>  : vm_offset_t va)
>  :          */
>  : 
>  :         TAILQ_FOREACH(pv, &pg->md.pv_list, pv_list) {
>  : +               if (pv->pv_flags & PVF_EXEC)
>  : +                       return;
>  :                         /* generate a count of the pv_entry uses */
>  :                 if (pv->pv_flags & PVF_WRITE) {
>  :                         if (pv->pv_pmap == pmap_kernel())
>  : 
>  : execution time of 'test' program is:
>  : mv78100-4# time ./test
>  : 5.000u 0.000s 0:05.40 99.8%     40+1324k 0+0io 0pf+0w
>  : 
>  : and without this path is:
>  : mv78100-4# time ./test
>  : 295.000u 0.000s 4:56.01 99.7%   40+1322k 0+0io 0pf+0w
>  : 
>  : 
>  : I think we need to handle executable pages in different way.
>
>  Agreed.  Why would we turn off caching for shared pages?  I can
>  understand read/write pages in a system that has a virtually index
>  cache with the classic cache aliasing problems as a workaround for
>  lousy hardware, but otherwise, this one has me scratching my head...
>
>  And if there's only one copy of 'test' running, why does it hit the
>  'shared' case for this code?
>
>  Warner

Could you do this instead:

+ 	int stop;
	TAILQ_FOREACH(pv, &pg->md.pv_list, pv_list) {
+               if (pv->pv_flags & PVF_EXEC)
+                       stop = 1;
			/* generate a count of the pv_entry uses */
		if (pv->pv_flags & PVF_WRITE) {
			if (pv->pv_pmap == pmap_kernel())
				kwritable++;
			else if (pv->pv_pmap == pm)
				uwritable++;
			writable++;
		}
		if (pv->pv_pmap == pmap_kernel())
			kentries++;
		else {
			if (pv->pv_pmap == pm)
				uentries++;
			entries++;
		}
	}

+	if (stop) {
+ 		if (writable && entries)
+ 			printf("fix results %d %d %d %d\n", kwritable,uwritable,
+ 				kentries, uentries);
+		return;
+	}

This would give counts to make sure there is not a logic error in fix_cache.

My best guess kwritable/kentry will be non-zero and says there is another
"dangling kernel allocation" somewhere. We allocate a page into a kernel
mapping and then either:

	1) we are not clearing the md.pv_kva entry all the time when
	   allocation is freed. I thought I was careful.

	2) someone is temporarilly allocating the page and not telling
	   pmap that they are done with it.

The information that this page is mapped to a kernel mapping is incorrectly
remembered, and the cache are turned off.

We could look in the free page code for a non-empty md.pv_kva entry to
test this theory.

md.pv_kva is set for kernel mappings, so the culiprit will be in the
pmap_kenter calls.

Thank-you.

--Mark Tinguely