1:1 Threading implementation.

[Daniel Eischen]

On Wed, 26 Mar 2003, Julian Elischer wrote:
> 
> On Wed, 26 Mar 2003, Jeff Roberson wrote:
> > >
> > > i.e. on creation of a new process, shced_newproc() is called
> > > and a KSE is added in there is the scheduler in question wants to use
> > > KSEs. If it doesn't, no KSE would be added, but it's still possible that
> > 
> > Yes, I think we need more sched hooks here as well.  Having only
> > sched_fork() makes things sort of gross.  We'll have to hook this all up
> > later.
> 
> I'll try get it hooked up "sooner rather than later".
> I think you can make 1:1 threads in the current system by doing:
> 
> kse_create(mbox, NEWGROUP); where the mbox points to the function you
> want to run and a new stack. The function just runs as normal, not
> knowing that it is atually a UTS thread. Since it never yields to
> another thread (by KSE terms) it never does any upcalls an voila.. 1:1
> threads. (I am sugesting that we don't need a new syscall to do this,
> or, at most a new entrypoint which ends up calling much of the same
> code.)

Right.  And if you translate this into the M:N library,
you just create your threads with PTHREAD_SCOPE_SYSTEM.

One of my unvoiced thoughts was that we could add a flag
or two to the KSE mailbox so that a scope system thread
doesn't need a separate stack.  Once one of these KSEs
(thread actually) blocks in the kernel, it stays there,
BUT, it can still awake from kse_thr_interrupt, kse_release,
etc, just that instead of an upcall it just returns
normally from those calls.  In this way, scope system
threads can be very low overhead and not need to enter
the UTS scheduler, yet they can still coexist with
scope process threads.

> Ok but htis breaks things for M:N threads as in M:N threads, teh mask
> would be stored "per process" (or at most per group) and the mask is the
> "logical OR" of all the masks for the threads in the group/process.
> Having a mask per thread and not having one for the bigger unit
> means that the masks for the threads must be updated regularly 
> (maybe at every kernel entry) to be the OR of the masks for ALL THE USER
> THREADS, which means that the UTS must do this explicitly.
> I'm not thrilled by all the extra work this is going to make for M:N
> threads. (Well at least this is my preliminary reading of it.)

No, please don't make the UTS deal with this, if that's the
case.

> > 
> > First off, it is many months away from being even beta quality.  I think
> > the UTS is far more complicated than you may realize.  There are all sorts
> > of synchronization issues that it was able to avoid before since only one
> > thread could run at any time and there essentially was no preemption.  It
> > now also has to deal with effecient scheduling decisions in a M:N model
> > that it didn't have to worry about before.
> 
> I'm not sure that teh issues there are as bad as you think.

I don't think it is as bad as that either.  The complexity
is on par with that of libc_r.

> > 
> > Then if you look at the number of system calls and shenanigans a UTS must
> > do to make proper scheduling decisions it doesn't look like such an
> > advantage.  I feel that the overhead of all the layers comes close to the
> > savings from doing some of it without entering the kernel.
> 
> So far it's not doing that much..

Yeah, I don't understand the above statement either.  It's lower
overhead than libc_r.  The only system calls it should be making
is to kse_release() when it has no more work to do (no runnable
threads) or possibly to kse_thr_wakeup() if it has to dispatch
signals to threads blocked in the kernel.  Time comes from the
mailbox so we don't even need to get the time of day.  The interfaces
were designed so that we _wouldn't_ have much syscall overhead.

-- 
Dan Eischen