[PATCH] hwpmc(4) changes to use 'mp_maxid' instead of 'mp_ncpus'.

[Joseph Koshy]

>  FreeBSD has been trying to not be quite as i386-centric as it used to be.  If
>  you look at other code in the kernel that handles per-cpu data such as UMA
>  you will see that it uses mp_maxid and CPU_ABSENT().  There are other places
>  in the kernel that are broken though (such as ndis(4)).

HWPMC is very x86 centric, for obvious reasons.

>  Yes, we can certainly export more stuff to userland.  The all_cpus mask would
>  be good as would a MI online_cpus mask, though at this point they would be
>  cpusets to handle > 32 rather than cpumask_t.  Note that machdep.hlt_cpus is
>  x86-only and would be superseded by a MI online_cpus mask.

Sure, an MI counter is a good idea.

>  > - Will sysctl hw.ncpus represent the count of present CPUs or will it
>  >   represent the maximum CPU id?
>
>  hw.ncpus is always mp_ncpus
>  kern.smp.cpus is also mp_ncpus
>  kern.smp.maxcpus is MAX_CPUS.

>  Userland can just iterate from 0 to kern.smp.maxcpus while handling absent
>  CPUs.  (For example, the kern.cp_time[] sysctl just writes out all 0's for
>  absent CPUs so that is how userland can determine an absent CPU in that
>  case.)

I thought of that.   For PMCTools use, using the proposed 'online_cpus' mask
would be a better option.   MAX_CPUS is a compile time value and could be
large, whereas most machines will have far fewer CPUs than that limit.
Why waste cycles needlessly?

Now it appears to me that in the scheme of things described
above one of mp_maxid and mp_ncpus is superfluous.

Here is the reasoning:

0) We need a compile time limit for the kernel; this is kern.smp.maxcpus.

1) A given machine has a maximum number of CPUs that can fit in it.
   This is usually <<= MAXCPUS.     Let us call this {MACHINE-MAX}.
   We need to scale kernel data structures based on {MACHINE-MAX}
   since using {MAXCPUS} is probably wasteful.  We cannot just count the
   current number of CPUS, as we do today, because more could be
   hotplugged in later.

2) At any given instant a subset of CPUs 0..{MACHINE_MAX} will be
    online. This would be tracked by the kern.smp.online_cpus/all_cpus
    bitmask.

Therefore we can use either a count (mp_ncpus) or a maximum id
(mp_maxid) to represent {MACHINE-MAX}, but  either one would do.

However, x86 MD code uses both, with newer code seeming to prefer
mp_maxid.  So I am puzzled.  There are far more uses of mp_ncpus
there though.

jk> Changing HWPMC and its userland before the base kernel itself
jk> changes does not seem to be the right thing to do.

jb>  While the userland intIerface is somewhat lacking, all of the in-kernel
jb>  infrastructure has been in place for at least the past 4 years,
and there is
jb>  no excuse for any in-kernel code not properly handling sparse CPU IDs.

I try keep userland, kernel and documentation associated with PmcTools
in sync.

Looking around, there appear to be lots of nits that need correction.
For one,  the kern.smp sysctl hierarchy is undocumented.


Thanks,
Koshy