uma_small_alloc from direct mapped memory.

[Jayachandran C.]

On Mon, Dec 6, 2010 at 3:24 PM, Juli Mallett <jmallett at freebsd.org> wrote:
> Hi JC,
>
> On Sun, Dec 5, 2010 at 22:39, Jayachandran C. <c.jayachandran at gmail.com> wrote:
>> The attached patch uses the mechanism used to allocate page table
>> pages for UMA small allocations too.  This will make sure that all the
>> small UMA allocations comes from KSEG0 pages in 32 bit and XKPHYS in
>> 64bit.
>>
>> This reduces TLB misses in kernel code significantly, and can give
>> major performance advantage for applications that spent a lot of time
>> in kernel.
>>
>> Please let me know your comments.
>
> It looks reasonable to me.  I've gone over the other unmerged changes
> related to the direct map in my Octeon branch and wonder if you would
> like to handle other expansions of use of the direct map, particularly
> on n64:
>
> o) Using the direct map for sendfile.  (Making sf_buf_alloc and
> sf_buf_free no-ops, like on sun4v, I think, rather than the whole
> freelist approach of e.g. sparc64, but I don't remember the details of
> why one implementation vs. the other at the moment.)
> o) Making uiomove_fromphys use XKPHYS on n64.  (I think that this will
> help our pipe performance a non-trivial amount, but it has been some
> time since I tested it.)
>
> If not I will try to make those changes in the near future, but I
> thought that (1) perhaps you had a reason for not making those changes
> with your n64 work in -CURRENT (2) since you're making related
> changes, you might like to do so, and also might be able to quantify
> what you feel the overall affect on performance to be.

I was planning to merge these changes. Now that the XLR platform code
changes and 8-STABLE merge is done - I hope to get some time to do the
rest of the n64 work. My rough todo list is :

- 64bit PTEs which will give >4GB phys address,
- 32 bit compat in n64 (merge from /user/jmallett/octeon)
- Complete merge from /user/jmallett/octeon - UIO, sf buf
- DDB/KDB fixes for n32/n64.
- 16K pagesize (there were some patches from SoC, but it crashed for me)

JC.