cpuset and affinity implementation
brooks at freebsd.org
Tue Feb 26 01:50:35 UTC 2008
On Mon, Feb 25, 2008 at 03:12:00PM -1000, Jeff Roberson wrote:
> On Mon, 25 Feb 2008, Brooks Davis wrote:
>> On Sun, Feb 24, 2008 at 05:38:37PM -1000, Jeff Roberson wrote:
>>> I have implemented a new api similar to processors sets on solaris. This
>>> allows you to assign processes to sets of cpus and dynamically change
>>> sets. This is useful for provisioning purposes to add and remove cpu
>>> resources for a particular process or group of processes. This new
>>> facility also supports binding secific threads to specific cpus which
>>> applications may want to do. At some point in the future this will be
>>> integrated with jail so you can restrict the cpus any jail is allowed to
>>> This api should not be considered final and the 'cpuset' tool is quite
>>> rough. This also only works with ULE and is unfortunately intertwined
>>> a big ULE patch I've been working on. The set management code is generic
>>> but 4BSD doesn't contain the hooks to actually constrain threads.
>> I took a look at the patch this morning. The API looks like it's
>> capable of doing what I need, at least at a first pass. It looks like I
>> should be able to implement the semantics currently employed by the Sun
>> Grid Engine scheduler on Irix systems.
>> The one thing I noticed that I found worrying was the recursion in
>> cpuset_(test)update(). It wasn't immediately clear to me there there
>> is anything to would prevent an arbitrarily deep hierarchy from being
>> created and blowing the kernel stack. I'm I missing something?
> Yes, presently it can never be more than 3 levels deep. Once we have jails
> the max would be 6 levels, unless you can make jails within jails.
> There is presently now way for the user to create a cpuset that is a subset
> of another set. So the three cpu sets are:
> 1) Root set - immutable, all cpus, may be root of jail in which case root
> outside of the jail can change the set.
> 2) cpuset - the set this process is a member of.
> 3) mask - the anonymous set that is applied to an individual thread.
OK. That makes sense. It would be useful from my perspective if
creating a root set (or an otherwise inescapable set) was not explicitly
tied to jails. I could see doing this either by extending the syscalls
or by introducing a more fine grained light weight virtualization as was
discussed in Milan. This would probably want to be a privileged operation
> Did you look at the userland tool at all? I think this needs the most
> improvement. I basically just made something that would allow me to pass
> every possible parameter to the api. Not exactly engineered for usability.
I glanced at it, but haven't really thought about what should/shouldn't be
there much. I'll try to do that in the next day or so.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Size: 187 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-current/attachments/20080226/90546dc7/attachment.pgp
More information about the freebsd-current