MAXCPU preparations

[Robert N. M. Watson]

On 29 Sep 2010, at 12:49, John Baldwin wrote:

> On Tuesday, September 28, 2010 6:24:32 pm Robert N. M. Watson wrote:
>> 
>> On 28 Sep 2010, at 19:40, Sean Bruno wrote:
>> 
>>>> If you go fully dynamic you should use mp_maxid + 1 rather than maxcpus.
>>> 
>>> I assume that mp_maxid is the new kern.smp.maxcpus?  Can you inject some
>>> history here so I can understand why one is "better" than the other?
>> 
>> So, unlike maxcpus, mp_maxid is in theory susceptible to races in a brave new world in which we support hotplug CPUs -- a brave new world we're 
> not yet ready for, however. If you do use mp_maxid, be aware you need to add one to get the size of the array -- maxcpus is the number of CPUs that 
> can be used, whereas mp_maxid is the highest CPU ID (counting from 0) that is used.
> 
> Hmm, I'm not sure that is true.  My feeling on mp_maxid is that it is the
> largest supported CPUID.  Platforms that support hotplug would need to set
> mp_maxid such that it has room for hotpluggable CPUs.  You don't want to
> go reallocate the UMA datastructures for every zone when a CPU is hotplugged
> for example.

Yes, we'll have to break one (or even both) of two current assumptions with the move to hotplug: contiguous in-use CPU IDs and mp_maxid representing the greatest possible CPU ID as a constant value. The former is guaranteed, but I wonder about the latter. On face value, you might say "oh, we know how many sockets there are", but if course, we don't know how many threads will arrive when a package is inserted.  For many subsystems, DPCPU will present a more than adequate answer for avoiding resizing, although not for low-level systems (perhaps such as UMA?). Likewise, on virtual machine platforms where hotplug actually might reflect a longer-term scheduling choice by the admin/hypervisor (i.e., resource partitioning), it may be harder to reason about what the future holds.

Robert