getaffinity/setaffinity and cpu sets.

[Jeff Roberson]

I have the following api working:

/*
  * Parameters for the level argument to getaffinity.
  */
#define CPU_LEVEL_SYS   1       /* All system cpus. */
#define CPU_LEVEL_AVAIL 2       /* Available cpus for which. */
#define CPU_LEVEL_WHICH 3       /* Actual mask for which. */

/*
  * Parameters for the which argument to {get,set}affinity.
  */
#define CPU_WHICH_TID   1       /* Specifies a thread id. */
#define CPU_WHICH_PID   2       /* Specifies a process id. */
#define CPU_WHICH_SET   3       /* Specifies a set id. */


Along with a CPU_CLR, CPU_COPY, CPU_ISSET, CPU_SET, CPU_ZERO for 
manipulating the sets.

int getaffinity(int level, int which, int id, int cpusetsize, long *mask);
int setaffinity(int which, int id, int cpusetsize, long *mask);

The get call has a notion of 'level' which allows us to fetch different 
masks.  The system set is all processors in the system.  The available set 
is the set of cpus available to the tid/pid in the 'which' argument.  An 
application would fetch the avail set and then potentially reduce it.

The setaffinity call doesn't have a level because the avail/sys sets are 
immutable.  You can only set things which can be specified by the which 
argument.

I also have a 'cpuset' command which can run a new program with a given 
cpu set, view and modify sets of arbitrary pids.  This is all working and 
I can supply patches if anyone is interested.  I have to implement 4BSD 
support before I can commit.

I have a proposal for solaris style processor sets which I think is simple 
and sufficient for most cases.  It involves the following new syscalls:

int     cpuset(void);
int     setcpuset(pid_t pid, int setid);
int     getcpuset(pid_t pid);

The notion would be that you can create a new numbered cpuset with 
cpuset().  You can modify or inspect its affinity with get/setaffinity 
above and the CPU_WHICH_SET argument.  The cpuset exists as long as there 
are members of the set.  Sort of like a process group or session.  The 
{get,set}cpuset calls can inspect or modify the state.

This set would not be modifiable by user processes or by processes in a 
jail.  It would create the restriction that differs between 'avail' and 
'sys' above.  Processors would be able to directly bind to any processor 
within the set.  Changing the set would apply to all processes in the set. 
The cpuset would be per-process while the mask is per-thread.  Sets 
involvement is inherited on fork().

In solaris sets can be named and have a more complete management api.  I'm 
not really interested in implementing all of that but I believe what I 
have outlined here would be subset of this and no code/syscalls would be 
wasted.

Comments?  Objections?  I'm fairly pleased with this arrangement now.

Thanks,
Jeff