svn commit: r208589 - head/sys/mips/mips

[Jayachandran C.]

On Wed, Jun 9, 2010 at 3:01 AM, Jayachandran C.
<c.jayachandran at gmail.com> wrote:
> On Tue, Jun 8, 2010 at 12:03 PM, Alan Cox <alc at cs.rice.edu> wrote:
>>
>> VM_FREEPOOL_DIRECT is used by at least amd64 and ia64 for page table pages
>> and small kernel memory allocations.  Unlike mips, these machines don't have
>> MMU support for a direct map.  Their direct maps are just a range of
>> mappings in the regular (kernel) page table.  So, unlike mips, accesses
>> through their direct map may still miss in the TLB and require a page table
>> walk.  VM_FREEPOOL_* is a way to increase the physical locality (or
>> clustering) of page allocations, so that, for example, page table page
>> accesses by the pmap on amd64 are less likely to miss in the TLB.  However,
>> it doesn't place a hard restriction on the range of physical addresses that
>> will be used, which you need for mips.
>>
>> The impact of this clustering can be significant.  For example, on amd64 we
>> use 2MB page mappings to implement the direct map.  However, old Opterons
>> only had 8 data TLB entries for 2MB page mappings.  For a uniprocessor
>> kernel running on such an Opteron, I measured an 18% reduction in system
>> time during a buildworld with the introduction of VM_FREEPOOL_DIRECT.  (See
>> the commit logs for vm/vm_phys.c and the comment that precedes the
>> VM_NFREEORDER definition on amd64.)
>>
>> Until such time as superpage support is ported to mips from the amd64/i386
>> pmaps, I don't think there is a point in having more than one VM_FREEPOOL_*
>> on mips.  And then, the point would be to reduce fragmentation of the
>> physical memory that could be caused by small allocations, such as page
>> table pages.
>
> Thanks for the detailed explanation.
>
> Also, after looking at the code again,  I think vm_phys_alloc_contig()
> can optimized not to look into segments which lie outside the area of
> interest. The patch is:
[...]
> This change, along with the vmparam.h changes for HIGHMEM, I think we
> should be able to use  vm_phys_alloc_contig() for page table pages (or
> have I again missed something fundamental?).

That patch was obviously wrong - many segments can map to a freelist
as the comment right above my change noted.

But the idea of eliminating freelists for which all the segments are
outside (low,high) may still be useful, will look at this some more.

JC.