pthread switch (was Odd KSE panic)

[Julian Elischer]

On Sun, 4 Jul 2004, Andrew Gallatin wrote:

> 
> Andrew Gallatin writes:
>  > 
>  > The problem happens when exiting.  When main application thread
>  > decides to exit, it does an ioctl into the driver to wakeup the
>  > sleeping worker thread.  The worker thread thread wakes up, and then
>  > exits, then the main thread closes his file descriptor and exits.
>  > 
>  > The problem I'm seeing is that I get a panic like the following when
>  > using KSE.  (A linux binary works fine, ioctls are translated..)
>  > 
> 
> The problem turned out to be that the worker thread was sleeping in
> cv_wait_sig() on a cv which was used elsewhere in the driver.  When I
> fixed this, pretty much everything got better.  I still don't
> understand exactly what happened.  I have no idea if the worker woke
> too early, or if the other place this cv was (mis)used was where the
> early wake happened.  (this would be where mx_free() is called).
> 
> Anyway, it no longer crashes the machine.   Sorry to have wasted your
> time.  
> 
> Now to figure out why libthr does pthread_cond_signal() in my scenario
> 47us faster than libpthread... (ULE, 1 HTT P4 running SMP kernel)
> 
> Scenario is that the mainline thread sleeps waiting for a packet
> (using pthread_cond_timedwait()) and the worker thread is asleep in
> kernel.  When a packet arrives, the worker wakes up, returns from the
> ioctl, does a pthread_cond_signal() to wakeup the mainline thread, and
> goes back into the kernel to sleep (via an ioctl).  This is the sort
> of scenario where I thought KSE would be faster than a 1:1 lib..

Actually it is the sort of scenario where it is not clear what is best
;-)

The mainline thread is asleep in userland. thus to become active, if
there is no running kernel thread to run it, one needs to be woken up
from the kernel and made to upcall to userspace to be assigned to run
the thread. This is pretty much what is happenning in M:M threads. (1:1 
is in my opinion a non threadded app :-) Except that in KSE there is
more overhead to do it.. One thing that could be tuned in KSE to make it
a lot faster is to reverse the order that the threads are run in the
following way:

------ currently:

Kernel thread 1 returns from waiting for the packet, and as user thread
A, signals (enters) the UTS to make user thread B runnable. The UTS
realises that there is no kernel thread currently running to run thread
B, and enters the kernel to wake one up, and returns to userland, in
order to re-enter thread A and go back into teh kernel to wait for the
next packet.

Kernel thread 2 is started up some time after it is awoken and enters
userland via an upcall and is assigned thread B to run.

This is all assuming there are 2 (or more CPUS).
If there is only one CPU then when user thread A sleeps in the kernel,
the kernel thread (1) is allowed to upcall back to the UTS and take on 
user thread B.


------ the changed version:

Kernel thread 1 returns from waiting for the packet, and as user thread
A, signals (enters) the UTS to make user thread B runnable. The UTS
realises that there is no kernel thread currently running to run thread
B it enters the kernel briefly to make another thread wake up and
upcall, and then suspends thread A, and takes on thread B.
Eventually th e upcall occurs and thread A is assigned to the newly
awoken thread and re-enters the kernel, waitign for the next kernel.

------------

Dan, can the 2nd version happen now? 
can Drew make it happen by assigning a higher priority to thread B?

> 
> Drew
> 
> _______________________________________________
> freebsd-threads at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-threads
> To unsubscribe, send any mail to "freebsd-threads-unsubscribe at freebsd.org"
>