Multiple page size support on FreeBSD?

[Cedric Blancher]

On 10 April 2013 22:27, Alfred Perlstein <bright at mu.org> wrote:
> On 4/10/13 1:09 PM, Andrew Duane wrote:
>>
>> Like all "performance" items (especially VM), it depends on the hardware
>> and the load. On systems with small TLBs it helps more than with large TLBs.
>> With software that needs access to lots of different areas the TLB gets more
>> traffic so large ones help more. The answer for your firefox browser box
>> with i386 is probably different from my compilation engine running MIPS, or
>> his web server running AMD.
>>
>> Back at Digital, we spent a lot of time trying to find the "one true
>> answer" to superpages, only to discover there wasn't one. We ended up with a
>> combination of automatic use from big allocations, a rarely used API to
>> advise for big TLBs, and some background work that coalesced when possible.
>
>
> Thank you Andrew.  I agree.  A good heuristic is great, but sometimes
> exposing the API unlocks some really awesome performance capabilities.
>
> It seems like both Digital and Sun went this route.
>
> I'm hoping we can do that as well.
>
> -Alfred
>
>
>>   ....................................
>> Andrew L. Duane
>> Resident Architect - AT&T Technical Lead
>> m   +1 603.770.7088
>> o   +1 408.933.6944 (2-6944)
>> skype: andrewlduane
>> aduane at juniper.net
>>
>>
>>
>> -----Original Message-----
>> From: owner-freebsd-hackers at freebsd.org
>> [mailto:owner-freebsd-hackers at freebsd.org] On Behalf Of Alfred Perlstein
>> Sent: Wednesday, April 10, 2013 4:00 PM
>> To: Benjamin Kaduk
>> Cc: Wojciech Puchar; Sebastian Feld; freebsd-hackers at freebsd.org
>> Subject: Re: Multiple page size support on FreeBSD?
>>
>> On 4/10/13 11:42 AM, Benjamin Kaduk wrote:
>>>
>>> On Wed, 10 Apr 2013, Wojciech Puchar wrote:
>>>
>>>>> How do your tests work?  Do you examine PTEs directly to check for
>>>>> superpages or are you relying on the vm.pmap.pde sysctls?
>>>>
>>>> the later.
>>>>
>>>> anyway - algorithm described on list - that heuristics detects
>>>> consecutive page access doesn't really help the urgent case - RANDOM
>>>> access to large amount of memory.
>>>
>>> The algorithm is not a heuristic based on consecutive accesses,
>>> promotion occurs when the entire superpage's worth of memory has
>>> actually been accessed.  If I remember correctly, the performance gain
>>> from superpages was only a few percent, so spending more time trying
>>> to decide when to use them would make the algorithm a net wash.
>>>
>>> You should really watch the talk I linked to if you're interested, it
>>> was quite interesting.
>>>
>>>> sequential access will get minimal improvement.
>>>>
>>>> IMHO the only way that really make sens is to add options to madvise
>>>> to give kernel information about usage.
>>>
>>> Maybe.
>>
>> It is cool that FreeBSD got this work via Alan Cox and the others that
>> contributed.
>>
>> I am wondering if it makes sense to have an explicit model.
>>
>> At one place, for a platform with high performance but a very small TLB,
>> we made it possible to explicitly request a large TLB for our process and it
>> made a BIG difference for performance.
>>
>> Sometimes being "general purpose" means that you can expose such low level
>> things for the user to tune instead of requiring them to fit the app to a
>> heuristic that may change.

As Sebastian said it might be worth (well, very worth) to implement
the Solaris userland API since there is software which uses it (since
Solaris >=9 ). Hey, even ksh93 uses MHA_MAPSIZE_STACK (to force 64k
pages when available;
http://www.opensource.apple.com/source/ksh/ksh-18/ksh/src/cmd/ksh93/sh/pmain.c?txt)
and the X11 xorg server does it, too.

Ced
-- 
Cedric Blancher <cedric.blancher at googlemail.com>
Institute Pasteur