svn commit: r224217 - in head/sys: amd64/include ia64/include mips/conf sys

[Sergey Kandaurov]

On 21 July 2011 17:20, John Baldwin <jhb at freebsd.org> wrote:
> On Thursday, July 21, 2011 8:37:26 am Sergey Kandaurov wrote:
>> On 21 July 2011 14:14, Attilio Rao <attilio at freebsd.org> wrote:
>> > 2011/7/20 Pan Tsu <inyaoo at gmail.com>:
>> >> Attilio Rao <attilio at FreeBSD.org> writes:
>> >>
>> >>> Author: attilio
>> >>> Date: Tue Jul 19 13:00:30 2011
>> >>> New Revision: 224217
>> >>> URL: http://svn.freebsd.org/changeset/base/224217
>> >>>
>> >>> Log:
>> >>>   Bump MAXCPU for amd64, ia64 and XLP mips appropriately.
>> >>>   From now on, default values for FreeBSD will be 64 maxiumum supported
>> >>>   CPUs on amd64 and ia64 and 128 for XLP. All the other architectures
>> >>>   seem already capped appropriately (with the exception of sparc64 which
>> >>>   needs further support on jalapeno flavour).
>> >>>
>> >>>   Bump __FreeBSD_version in order to reflect KBI/KPI brekage introduced
>> >>>   during the infrastructure cleanup for supporting MAXCPU > 32. This
>> >>>   covers cpumask_t retiral too.
>> >>>
>> >>>   The switch is considered completed at the present time, so for
> whatever
>> >>>   bug you may experience that is reconducible to that area, please
> report
>> >>>   immediately.
>> >>>
>> >>>   Requested by:       marcel, jchandra
>> >>>   Tested by:  pluknet, sbruno
>> >>>   Approved by:        re (kib)
>> >>>
>> >>> Modified:
>> >>>   head/sys/amd64/include/param.h
>> >>>   head/sys/ia64/include/param.h
>> >>>   head/sys/mips/conf/XLP
>> >>>   head/sys/mips/conf/XLP64
>> >>>   head/sys/mips/conf/XLPN32
>> >>>   head/sys/sys/param.h
>> >>>
>> >>> Modified: head/sys/amd64/include/param.h
>> >>>
> ==============================================================================
>> >>> --- head/sys/amd64/include/param.h    Tue Jul 19 12:41:57 2011
>  (r224216)
>> >>> +++ head/sys/amd64/include/param.h    Tue Jul 19 13:00:30 2011
>  (r224217)
>> >>> @@ -65,7 +65,7 @@
>> >>>
>> >>>  #if defined(SMP) || defined(KLD_MODULE)
>> >>>  #ifndef MAXCPU
>> >>> -#define MAXCPU               32
>> >>> +#define MAXCPU               64
>> >>>  #endif
>> >>>  #else
>> >>>  #define MAXCPU               1
>> >>
>> >> Do you plan to bump MEMSTAT_MAXCPU, too?
>> >>
>> >>  $ vmstat -z
>> >>  vmstat: memstat_sysctl_uma: Too many CPUs
>> >>  $ vmstat -m
>> >>  vmstat: memstat_sysctl_malloc: Too many CPUs
>> >>
>> >>  $ sysctl kern. | grep smp.\*cpus
>> >>  kern.smp.maxcpus: 64
>> >>  kern.smp.cpus: 2
>> >>
>> >
>> > Jeeeez, we seriously need to fix this getting rid of the static values.
>> >
>> > Anyway, can you try the following patch?:
>> > http://www.freebsd.org/~attilio/memstat_maxcpu.diff
>> >
>> > It is going to add some memory overhead for i386 case.
>> >
>>
>> Something like this should work (vmstat -z, vmstat -m both work).
>> It gets rid of MEMSTAT_MAXCPU at the expense of malloc() at runtime.
>> http://plukky.net/~pluknet/patches/libmemstat_nomaxcpu.diff
>>
>> Probably it should work with maxid, instead of maxcpu to save some memory.
>> Though, using maxcpu is more safe.
>
> Actually, I would prefer that it use mp_maxid as that is the general variable
> things should use.  mp_maxcpus is a concession for the few places that may
> need to know the MAXCPUS value (e.g. if using libkvm to access a structure in
> a crashdump or live kernel that has a member array with MAXCPU elements).
> Code that just wants to allocate memory to hold per-CPU data should use
> mp_maxid whenever possible.
>

Hi,

I changed the patch to use mp_maxid wherever possible.
http://plukky.net/~pluknet/patches/libmemstat_nomaxcpu.2.diff

To summarize:

1) malloc stats
kern.malloc_stats uses internally MAXCPU, and we have to query MAXCPU
from kernel, too. See kern/kern_malloc.c:sysctl_kern_malloc_stats():
849    mtsh.mtsh_maxcpus = MAXCPU;

1a) memstat_sysctl_malloc()
left unchanged, sysctl kern.smp.maxcpus
1b) memstat_kvm_malloc()
left unchanged, _mp_maxcpus symbol

2) uma stats
vm.zone_stats uses (mp_maxid + 1), vm/uma_core.c:sysctl_vm_zone_stats():
3247    ush.ush_maxcpus = (mp_maxid + 1);

2a) memstat_sysctl_uma()
Switched to query sysctl kern.smp.maxid
2b) memstat_kvm_uma()
left unchanged,  _mp_maxid symbol

So, there's only one change in memstat_sysctl_uma().
A bad side of things is that libmemstat() now knows these vm_zone
and malloc_stats internals.
As Robert suggested me on IRC to query maxcpu value from
uma_stream_header and malloc_type_stream_header structures
respectively to be independent from kernel details.

-- 
wbr,
pluknet