PTHREAD_CANCEL_DEFERRED

[David Xu]

My idea is to always let close() syscall run, until it will be
blocked by lower layer, e.g tty or socket layer, as manual of close()
said it always release the file descriptor despite whether
it returns EINTR or EWOULDBLOCK, when it is going to sleep,
a flag will cause it to abort the sleep.
if the thread does not found the flag at that time,
a latter signal SIGCANCEL sent by pthread_cancel will unblock it,
this is how signal works.

A flag called TDP_WAKEUP is checked in sleepq_catch_signals(),
if the flag is set, the sleepq_catch_signals return EINTR.

In fact, a similar mechanism was already implemented in libthr's
condition variable, if you carefully read the code in libthr's
sigcancel_handler(), there are following lines:

static void
sigcancel_handler(int sig __unused,
         siginfo_t *info __unused, ucontext_t *ucp __unused)
{
         struct pthread *curthread = _get_curthread();

         if (curthread->cancel_defer && curthread->cancel_pending)
                 thr_wake(curthread->tid);




---
If cancellation is in defer mode and canceling request is pending,
the above code turns on kernel thread's flag TDP_WAKEUP, if now a
syscall is started, if kernel code finds this flag, it won't sleep
and immediately return to userland. However the cancel_defer now
is only used by condition variable implementation. it can be used
by every cancellation point functions if sleepq_catch_signals
checks the flag.

As current we do, if the sigcancel_handler found the thread is
in asynchronous mode, we immediately kill the thread.

Please read the code in kernel function kern_umtx.c:do_cv_wait(),
it checks the flag, but I think it is better to make the scope
wider, and just let subr_sleepq.c: sleepq_catch_signals() check it,
so we can do every cancellation point in cooperative way.

Once a cancellation point syscall returns, the code in libthr's
function _thr_cancel_leave_defer() checks cancellation request,
and cancel the thread, yes, it calls pthread_exit to kill the
current thread.
it was already implemented in libthr's cond_wait_common() function.

If we check TDP_WAKEUP in sleepq_catch_signals, then the close()
problem will be resolved, we even can check return value of
very cancellation point syscall, for example, if a recv/send
syscall return value greater than zero, we may not kill the thread,
this is really fun.

so the close() syscall wrapper in libthr would looks like this:

   1. enter cancellation point
   2. call syscall __sys_close()
   3. leave cancalltion point, check cancellation request,
      if set, kill the thread.

Finally, I only need to slightly tweak libthr to fix the problem you
have found, I don't need to change kernel's signal code at all,
no performance problem for thread cancellation. I am working on the
patch.

Regards,
David Xu