git: 72ed4d012ea9 - stable/15 - umtx: do not sleep on an unowned mutex after a spurious CAS failure

The branch stable/15 has been updated by pkubaj:

URL: https://cgit.FreeBSD.org/src/commit/?id=72ed4d012ea9a0ad66848f813360b0bcf20c213e

commit 72ed4d012ea9a0ad66848f813360b0bcf20c213e
Author:     Piotr Kubaj <pkubaj@FreeBSD.org>
AuthorDate: 2026-09-03 14:08:45 +0000
Commit:     Piotr Kubaj <pkubaj@FreeBSD.org>
CommitDate: 2026-09-18 13:34:25 +0000

    umtx: do not sleep on an unowned mutex after a spurious CAS failure
    
    On ll/sc architectures casueword32() may report a spurious
    store-conditional failure (reservation lost to an interrupt, preemption,
    or another CPU touching the same reservation granule), and this is
    indistinguishable from a genuine comparison mismatch: both return 1.
    That is intentional since D20772 and documented in casueword(9) ("The
    store can fail on load-linked/store-conditional architectures."), so
    callers must cope.
    
    do_lock_normal() does not fully cope.  When the initial
    UMUTEX_UNOWNED -> id acquire CAS fails spuriously, the observed owner is
    still UMUTEX_UNOWNED, so neither the UMUTEX_CONTESTED branch nor the
    real-owner case applies, and execution falls through past the "rv == 1
    but not contested, likely store failure" comment into the sleep path.
    There, the contested-bit CAS (expecting the observed owner, i.e.
    UMUTEX_UNOWNED) succeeds because the mutex really is unowned, stamping
    m_owner = UMUTEX_CONTESTED with no owner tid, and the thread sleeps on
    "umtxn" forever: nobody owns the mutex, so no unlock and no wakeup ever
    arrive.  In _UMUTEX_TRY mode the same situation returns a false EBUSY
    for a free mutex.
    
    Treat an observed owner of UMUTEX_UNOWNED like UMUTEX_CONTESTED: try to
    acquire the mutex, setting the contested bit, instead of falling through
    to the sleep path.  rv == 1 with the observed value equal to the
    expected value can only mean a spurious store failure, so the mutex is
    free.  If the acquire CAS fails again, the outer loop restarts and
    re-evaluates ownership.  The contested bit set with no waiters present
    only costs the matching unlock one trip through the kernel.
    
    This was hit in practice on powerpc64le (POWER9): the Swift runtime's
    Synchronization.Mutex issues _umtx_op(UMTX_OP_MUTEX_LOCK) directly with
    no userspace fast path, so an uncontended lock of an unowned mutex runs
    the kernel CAS exactly where a spurious failure deadlocks
    (single-threaded process parked on "umtxn" with m_owner == 0x80000000,
    observed as Foundation.Process.run() hanging).  libthr mostly masks the
    bug because pthread_mutex_lock() enters the kernel only when there is a
    real owner that will eventually issue a wakeup.
    
    The mechanism was confirmed with an experimental powerpc kernel that
    instead retried the ll/sc sequence inside casueword32()/casueword();
    that also eliminated the hang, but is not proposed here since the
    single-attempt semantics of casueword(9) are intentional.
    
    Reviewed by:    kib
    MFC after:      2 weeks
    Differential Revision:  https://reviews.freebsd.org/D59338
    
    (cherry picked from commit c5d9485e7d7faeba1019301ed83a781dac2d0cb0)
---
 sys/kern/kern_umtx.c | 20 ++++++++++++--------
 1 file changed, 12 insertions(+), 8 deletions(-)

diff --git a/sys/kern/kern_umtx.c b/sys/kern/kern_umtx.c
index ab6e578ab380..77ce0f97bb4c 100644
--- a/sys/kern/kern_umtx.c
+++ b/sys/kern/kern_umtx.c
@@ -1423,19 +1423,23 @@ do_lock_normal(struct thread *td, struct umutex *m, uint32_t flags,
 			}
 
 			/*
-			 * If no one owns it but it is contested try
-			 * to acquire it.
+			 * If no one owns it, but it is contested or
+			 * the CAS above failed spuriously (possible
+			 * on ll/sc architectures), try to acquire it.
+			 * Sleeping would be forever in the spurious
+			 * case: no owner exists to wake us.
 			 */
 			MPASS(rv == 1);
-			if (owner == UMUTEX_CONTESTED) {
-				rv = casueword32(&m->m_owner,
-				    UMUTEX_CONTESTED, &owner,
-				    id | UMUTEX_CONTESTED);
+			if (owner == UMUTEX_CONTESTED ||
+			    owner == UMUTEX_UNOWNED) {
+				rv = casueword32(&m->m_owner, owner,
+				    &owner, id | UMUTEX_CONTESTED);
 				/* The address was invalid. */
 				if (rv == -1)
 					return (EFAULT);
 				if (rv == 0) {
-					MPASS(owner == UMUTEX_CONTESTED);
+					MPASS(owner == UMUTEX_CONTESTED ||
+					    owner == UMUTEX_UNOWNED);
 					return (0);
 				}
 				if (rv == 1) {
@@ -1451,7 +1455,7 @@ do_lock_normal(struct thread *td, struct umutex *m, uint32_t flags,
 				continue;
 			}
 
-			/* rv == 1 but not contested, likely store failure */
+			/* rv == 1 with a real owner, fall through to sleep. */
 			rv = thread_check_susp(td, false);
 			if (rv != 0)
 				return (rv);