svn commit: r224217 - in head/sys: amd64/include ia64/include mips/conf sys

[Sean Bruno]

On Wed, 2011-07-27 at 05:45 -0700, John Baldwin wrote:
> On Wednesday, July 27, 2011 2:24:21 am Sergey Kandaurov wrote:
> > On 21 July 2011 17:20, John Baldwin <jhb at freebsd.org> wrote:
> > > On Thursday, July 21, 2011 8:37:26 am Sergey Kandaurov wrote:
> > >> On 21 July 2011 14:14, Attilio Rao <attilio at freebsd.org> wrote:
> > >> > 2011/7/20 Pan Tsu <inyaoo at gmail.com>:
> > >> >> Attilio Rao <attilio at FreeBSD.org> writes:
> > >> >>
> > >> >>> Author: attilio
> > >> >>> Date: Tue Jul 19 13:00:30 2011
> > >> >>> New Revision: 224217
> > >> >>> URL: http://svn.freebsd.org/changeset/base/224217
> > >> >>>
> > >> >>> Log:
> > >> >>>   Bump MAXCPU for amd64, ia64 and XLP mips appropriately.
> > >> >>>   From now on, default values for FreeBSD will be 64 maxiumum supported
> > >> >>>   CPUs on amd64 and ia64 and 128 for XLP. All the other architectures
> > >> >>>   seem already capped appropriately (with the exception of sparc64 which
> > >> >>>   needs further support on jalapeno flavour).
> > >> >>>
> > >> >>>   Bump __FreeBSD_version in order to reflect KBI/KPI brekage introduced
> > >> >>>   during the infrastructure cleanup for supporting MAXCPU > 32. This
> > >> >>>   covers cpumask_t retiral too.
> > >> >>>
> > >> >>>   The switch is considered completed at the present time, so for
> > > whatever
> > >> >>>   bug you may experience that is reconducible to that area, please
> > > report
> > >> >>>   immediately.
> > >> >>>
> > >> >>>   Requested by:       marcel, jchandra
> > >> >>>   Tested by:  pluknet, sbruno
> > >> >>>   Approved by:        re (kib)
> > >> >>>
> > >> >>> Modified:
> > >> >>>   head/sys/amd64/include/param.h
> > >> >>>   head/sys/ia64/include/param.h
> > >> >>>   head/sys/mips/conf/XLP
> > >> >>>   head/sys/mips/conf/XLP64
> > >> >>>   head/sys/mips/conf/XLPN32
> > >> >>>   head/sys/sys/param.h
> > >> >>>
> > >> >>> Modified: head/sys/amd64/include/param.h
> > >> >>>
> > > ==============================================================================
> > >> >>> --- head/sys/amd64/include/param.h    Tue Jul 19 12:41:57 2011
> > >  (r224216)
> > >> >>> +++ head/sys/amd64/include/param.h    Tue Jul 19 13:00:30 2011
> > >  (r224217)
> > >> >>> @@ -65,7 +65,7 @@
> > >> >>>
> > >> >>>  #if defined(SMP) || defined(KLD_MODULE)
> > >> >>>  #ifndef MAXCPU
> > >> >>> -#define MAXCPU               32
> > >> >>> +#define MAXCPU               64
> > >> >>>  #endif
> > >> >>>  #else
> > >> >>>  #define MAXCPU               1
> > >> >>
> > >> >> Do you plan to bump MEMSTAT_MAXCPU, too?
> > >> >>
> > >> >>  $ vmstat -z
> > >> >>  vmstat: memstat_sysctl_uma: Too many CPUs
> > >> >>  $ vmstat -m
> > >> >>  vmstat: memstat_sysctl_malloc: Too many CPUs
> > >> >>
> > >> >>  $ sysctl kern. | grep smp.\*cpus
> > >> >>  kern.smp.maxcpus: 64
> > >> >>  kern.smp.cpus: 2
> > >> >>
> > >> >
> > >> > Jeeeez, we seriously need to fix this getting rid of the static values.
> > >> >
> > >> > Anyway, can you try the following patch?:
> > >> > http://www.freebsd.org/~attilio/memstat_maxcpu.diff
> > >> >
> > >> > It is going to add some memory overhead for i386 case.
> > >> >
> > >>
> > >> Something like this should work (vmstat -z, vmstat -m both work).
> > >> It gets rid of MEMSTAT_MAXCPU at the expense of malloc() at runtime.
> > >> http://plukky.net/~pluknet/patches/libmemstat_nomaxcpu.diff
> > >>
> > >> Probably it should work with maxid, instead of maxcpu to save some memory.
> > >> Though, using maxcpu is more safe.
> > >
> > > Actually, I would prefer that it use mp_maxid as that is the general variable
> > > things should use.  mp_maxcpus is a concession for the few places that may
> > > need to know the MAXCPUS value (e.g. if using libkvm to access a structure in
> > > a crashdump or live kernel that has a member array with MAXCPU elements).
> > > Code that just wants to allocate memory to hold per-CPU data should use
> > > mp_maxid whenever possible.
> > >
> > 
> > Hi,
> > 
> > I changed the patch to use mp_maxid wherever possible.
> > http://plukky.net/~pluknet/patches/libmemstat_nomaxcpu.2.diff
> > 
> > To summarize:
> > 
> > 1) malloc stats
> > kern.malloc_stats uses internally MAXCPU, and we have to query MAXCPU
> > from kernel, too. See kern/kern_malloc.c:sysctl_kern_malloc_stats():
> > 849    mtsh.mtsh_maxcpus = MAXCPU;
> > 
> > 1a) memstat_sysctl_malloc()
> > left unchanged, sysctl kern.smp.maxcpus
> > 1b) memstat_kvm_malloc()
> > left unchanged, _mp_maxcpus symbol
> > 
> > 2) uma stats
> > vm.zone_stats uses (mp_maxid + 1), vm/uma_core.c:sysctl_vm_zone_stats():
> > 3247    ush.ush_maxcpus = (mp_maxid + 1);
> > 
> > 2a) memstat_sysctl_uma()
> > Switched to query sysctl kern.smp.maxid
> > 2b) memstat_kvm_uma()
> > left unchanged,  _mp_maxid symbol
> > 
> > So, there's only one change in memstat_sysctl_uma().
> > A bad side of things is that libmemstat() now knows these vm_zone
> > and malloc_stats internals.
> > As Robert suggested me on IRC to query maxcpu value from
> > uma_stream_header and malloc_type_stream_header structures
> > respectively to be independent from kernel details.
> 
> Looks good to me, thanks!
> 

Aye +1 here.  I'll run this over to peter@ just to be sure though.  I'm
pretty sure that this will just work.

I'll ask HP for their big machine again to validate.

Sean