Unlocking a robust mutex in a cleanup handler

[Konstantin Belousov]

On Tue, Dec 06, 2016 at 03:17:41PM +0100, Dimitri Staessens wrote:
> Dear Konstantin,
> 
> thanks for your immediate response. Please find attached a minimal code 
> example. I do hope I'm sending in the correct format, I'm new to the 
> community.
> 
> The test creates an integer, robust mutex and condition variable in 
> shared memory. A thread blocks on that condition variable with the 
> associated mutex. After one second, the main thread cancels the blocking 
> thread.
> 
> I compile as follows:
> 
> gcc robust_test.c -lpthread -lrt -o robust_test
> 
> and run
> 
> ./robust_test
> 
> On linux it gives the following output:
> [dstaesse at phoneutria]$ ./robust_test
> Initializing...
> Starting thread...
> Sleeping for one second...
> Thread started...
> Cancelling thread...
> Thread finished.
> Bye.
> 
> On FreeBSD I get the following:
> $ ./robust_test
> Initializing...
> Starting thread...
> Sleeping for one second...
> Thread started...
> Cancelling thread...
> Fatal error 'inact_mtx enter' at line 188 in file 
> /usr/src/lib/libthr/thread/thr_mutex.c (errno = 0)
> Abort trap (core dumped)

Try this patch.  It worked for me.
It is enough to patch and then rebuild only libthr:
cd /usr/src
patch -p1 <patch
(cd lib/libthr && make WITHOUT_TESTS=yes all install)

diff --git a/lib/libthr/thread/thr_cond.c b/lib/libthr/thread/thr_cond.c
index 506b8eca9e7..64d075ca06f 100644
--- a/lib/libthr/thread/thr_cond.c
+++ b/lib/libthr/thread/thr_cond.c
@@ -224,16 +224,26 @@ cond_wait_kernel(struct pthread_cond *cvp, struct pthread_mutex *mp,
 		 * state and unlock the mutex without making the state
 		 * consistent and the state will be unrecoverable.
 		 */
-		if (error2 == 0 && cancel)
+		if (error2 == 0 && cancel) {
+			if (robust) {
+				_mutex_leave_robust(curthread, mp);
+				robust = false;
+			}
 			_thr_testcancel(curthread);
+		}
 
 		if (error == EINTR)
 			error = 0;
 	} else {
 		/* We know that it didn't unlock the mutex. */
 		_mutex_cv_attach(mp, recurse);
-		if (cancel)
+		if (cancel) {
+			if (robust) {
+				_mutex_leave_robust(curthread, mp);
+				robust = false;
+			}
 			_thr_testcancel(curthread);
+		}
 		error2 = 0;
 	}
 	if (robust)