NUMA, cpuset and malloc

[John Baldwin]

On Monday, April 22, 2013 12:29:29 pm Freddie Cash wrote:
> On Mon, Apr 22, 2013 at 8:32 AM, John Baldwin <jhb at freebsd.org> wrote:
> 
> > On Saturday, April 20, 2013 6:43:26 pm Robert Waksmundzki wrote:
> > > On NUMA systems allocated memory is striped across local and non-local
> > banks
> > in order to have consistent performance in case the task is rescheduled to
> > a
> > different CPU socket.
> > > When a process is pinned to a single CPU socket with cpuset having the
> > memory allocator prefer local banks would probably improve performance.
> > Default system behavior would stay the same and the optimization would
> > only be
> > triggered on big multi socket systems when administrator used cpuset
> > (command
> > mostly used for performance optimization anyway).
> > >
> > > Is this something currently implemented in FreeBSD? Is this even a good
> > idea?
> >
> > You can get something sort of like this by enabling NUMA in your kernel
> > (9.0
> > and later) and always pinning your processes with cpuset.  (The simple NUMA
> > bits always allocate memory in the memory domain the current thread is
> > running in at the time of the fault.)
> >
> 
> How does one enable NUMA?
> 
> A "grep -i numa /usr/src/sys/conf/NOTES /usr/src/sys/amd64/conf/NOTES"
> turns up 0 hits for both 9-STABLE r248547 and 10-CURRENT (April 11, used
> svnup so no way to get the exact revision number, that I know of).
> 
> Or, is it enabled automatically?

You have to chagne the VM_NDOMAIN setting.  In recent HEAD and 9-stable
you can do it in the kernel config (options VM_NDOMAIN=4 for example).
In older HEAD and 9 you have to edit sys/amd64/include/vmparam.h or
sys/i386/include/vmparam.h and change VM_NDOMAIN before building your
kernel.

-- 
John Baldwin