Use of contiguous physical memory in cxgbe driver

[John Baldwin]

On Tuesday, February 11, 2014 6:30:04 pm Adrian Chadd wrote:
> On 11 February 2014 10:48, John Baldwin <jhb at freebsd.org> wrote:
> > On Thursday, January 23, 2014 2:32:51 am Adrian Chadd wrote:
> >> It's about time we taught the physmem allocator to be more conducive
> >> to physically contiguous allocations.
> >>
> >> A server with gigabytes of memory should be able to keep a couple tens
> >> of megabytes of 64k sized allocation chunks around for exactly this.
> >
> > Alan Cox already taught the physmem allocator to do this for superpages.
> > However, this change was part of the superpages changes, so you can't 
count
> > experiences from machines older than about 7.2 when evaluating the
> > effectiveness of the new allocator.
> 
> the problem is that we don't have pressure to _not_ fragment the
> physical memory from the allocator, so all of the memory ends up
> getting very fragmented versy quickly.
> 
> the physmem superpage stuff stops being viable after a short while of
> say, "pound lots of packets around from vm objects" workload, because
> suddenly we end up chewing through all of physical memory with pages
> extremely quickly.

Is this because UMA keeps lots of mbufs cached in your workload?  The physmem 
buddy allocator certainly seeks to minimize fragmentation.  However, it can't 
go yank memory out of UMA caches to do so.

-- 
John Baldwin