1 gig superpages

[John Baldwin]

On Monday, November 10, 2014 11:15:09 am Sebastian Kuzminsky wrote:
> Hello hackers, I'm announcing the availability of a branch adding support 
for 1 GB superpages to FreeBSD.
> 
> https://github.com/Seb-LineRate/freebsd/commits/seb/stable-10/1-gig-pages
> 
> The branch is based on work done by Line Rate Systems and F5 Networks, and 
used in our LROS load-balancing product.
> 
> Our product is based on FreeBSD 9.1; the branch I linked to above is our 1 
gig page support rebased onto stable/10.  I probably messed something up in 
the rebase, as lots of things changed both in pmap and vm since 9.1.  There 
are also a handful of commits that i haven't gotten to yet, but they are less 
consequential - just performance improvements to the buddy allocator.  I hope 
to push those over the next few days.
> 
> It should be relatively easy to rebase the branch onto Current.
> 
> 
> This is a work in progress, and I would appreciate feedback and comments.

So my initial thoughts from having looked at this very briefly is that this
is a bit hackish.  In particular, the reservation system already supports
multiple levels of reservations so that you could have a separate reservation
layer for 1GB pages.

However, that alone doesn't get you exactly what you want, which is that you
want to guarantee a specific page size.  This is also something that would be
nice to have for 2MB pages as well, and I have talked a bit about that with
Alan in the past.

What I do think would be useful would be to have a new mmap flag which 
requests that a mapping not use demotion/promotion but fully use any
reservations it makes.  You could also have it fail if it can't get 
reservations for the entire range.  Alan suggests to call this MAP_HUGETLB to 
match Linux since it would provide similar semantics.

If you then add 1GB pages as an second reservation level on amd64 and make the 
semantics of MAP_HUGETLB such that it uses the largest reservations possible 
for the mapping size (so a request for 1G uses 1G page instead of 2M pages), 
then that would give you what you want without having various 1G-specific 
functions scattered in MI code, etc.  It will also be more useful for other 
platforms going forward (some of which support multiple page sizes that aren't 
just the 'trim a tree layer' PSE-style you have on x86).

-- 
John Baldwin