uma_small_alloc from direct mapped memory.

[Juli Mallett]

Hi JC,

On Sun, Dec 5, 2010 at 22:39, Jayachandran C. <c.jayachandran at gmail.com> wrote:
> The attached patch uses the mechanism used to allocate page table
> pages for UMA small allocations too.  This will make sure that all the
> small UMA allocations comes from KSEG0 pages in 32 bit and XKPHYS in
> 64bit.
>
> This reduces TLB misses in kernel code significantly, and can give
> major performance advantage for applications that spent a lot of time
> in kernel.
>
> Please let me know your comments.

It looks reasonable to me.  I've gone over the other unmerged changes
related to the direct map in my Octeon branch and wonder if you would
like to handle other expansions of use of the direct map, particularly
on n64:

o) Using the direct map for sendfile.  (Making sf_buf_alloc and
sf_buf_free no-ops, like on sun4v, I think, rather than the whole
freelist approach of e.g. sparc64, but I don't remember the details of
why one implementation vs. the other at the moment.)
o) Making uiomove_fromphys use XKPHYS on n64.  (I think that this will
help our pipe performance a non-trivial amount, but it has been some
time since I tested it.)

If not I will try to make those changes in the near future, but I
thought that (1) perhaps you had a reason for not making those changes
with your n64 work in -CURRENT (2) since you're making related
changes, you might like to do so, and also might be able to quantify
what you feel the overall affect on performance to be.

Thanks again for all of your work in this area!

Juli.