cpuset and affinity implementation

[Daniel Eischen]

On Mon, 25 Feb 2008, Jeff Roberson wrote:

> On Mon, 25 Feb 2008, Daniel Eischen wrote:
>
>> On Mon, 25 Feb 2008, Jeff Roberson wrote:
>> 
>>> On Mon, 25 Feb 2008, Alfred Perlstein wrote:
>>> 
>>>> Jeff, this is very cool.  I do have one issue though:
>>>> 
>>>> + * A thread may not be assigned to a a group seperate from other threads 
>>>> in
>>>> + * the process.  This is to remove ambiguity when the setid is queried 
>>>> with
>>>> + * a pid argument.  There is no other technical limitation.
>>>> 
>>>> Am I understanding things correctly such that within a process there
>>>> can only be one "set"?
>>>> 
>>>> If so this restricts some of the benifits you get with sets and
>>>> binding.
>>>> 
>>>> An example would be some sort of system with multiple CPUs where some
>>>> are assigned specifically for pseudo-realtime processing and others are 
>>>> for
>>>> general control things such as cli, stats, etc.
>>>> 
>>>> In our case we would like to be able to run some threads on specific
>>>> cpu sets, and other threads to be run anywhere on the control CPUs.
>>>> 
>>>> Can this be done with this API?
>>> 
>>> Individual threads can be bound to any cpu or group of cpus within the 
>>> set. So if you just make a set that includes all cpus in the system you 
>>> can then bind your realtime threads to specific cpus and the other threads 
>>> to the remainder.  You will have to specifically bind each thread however.
>>> 
>>> The reason individual threads can't be assigned to groups is because 
>>> cpuset_getid() for a pid wouldn't make sense then and I expect 
>>> administrators to be mostly interested in managing groups of processes.
>> 
>> If the administrator sets up a set of CPUs specifically for
>> real-time and another set for non-real-time, you may want to
>> bind some threads to the real-time set, and leave other threads
>> unbound (or even bound to the non-real-time set).  In this
>> case, I think cpuset_getid() should either return the default
>> cpuset of all cpus in the system, or the last cpuset to
>> which the process was bound.
>> 
>> But regardless, I think binding a thread to a different
>> processor set should be allowed and should override its
>> inherent binding of the process' processor set.
>> Hmm, I guess in this case, a subsequent binding of the
>> process to a processor set should probably override any
>> per-thread bindings.
>
> I think we're getting into complex corner cases here which will only confuse 
> the api and administrators.  I don't expect administrators will want to set 
> groups to individual threads.  How would he even identify the individual 
> thread?  And if he did, he could just as easily set masks on that thread 
> along with others in the process.
>
> I'm already a little nervous about how complicated this will be for 
> programmers.   If we allowed each thread in a pid to be in its own set, we'd 
> have to make cpuset_getid() return an array of ids.  I definitely don't want 
> to do that.

Solaris does seem to allow this BTW.

-- 
DE