git: 1744df209f7f - stable/15 - critical_exit_preempt(): Re-check for preemption locked

From: Olivier Certner <olce_at_FreeBSD.org>
Date: Thu, 19 Nov 2026 15:32:14 UTC
The branch stable/15 has been updated by olce:

URL: https://cgit.FreeBSD.org/src/commit/?id=1744df209f7f6afa353574a2b91da74faefa9831

commit 1744df209f7f6afa353574a2b91da74faefa9831
Author:     Olivier Certner <olce@FreeBSD.org>
AuthorDate: 2026-10-13 13:20:43 +0000
Commit:     Olivier Certner <olce@FreeBSD.org>
CommitDate: 2026-11-19 15:31:49 +0000

    critical_exit_preempt(): Re-check for preemption locked
    
    Once we have the thread lock, we are in a spinlock section and thus will
    not be preempted, which ensures stability of 'td_owepreempt'.  Check it
    again for the improbable case where we were already preempted before
    entering that section (after 'td_critnest' reached 0 in critical_exit()
    but before clearing 'td_owepreempt', because of an IPI_PREEMPT received
    in the meantime or any interrupt whose handler uses critical sections),
    in which case there is no need to switch to another thread.
    
    It is however still necessary to temporarily raise 'td_critnest' around
    the thread_lock() call, and that is not an optimization contrary to what
    the old comment stated.  As explained in the new comment, thread_lock()
    can lead to entering and exiting a spinlock section multiple times,
    which would cause recursion with critical_exit_preempt() in some
    circumstances.
    
    Reviewed by:    kib, markj
    MFC after:      3 days
    Sponsored by:   The FreeBSD Foundation
    Differential Revision:  https://reviews.freebsd.org/D60860
    
    (cherry picked from commit 28c31242758323499eaf8af1f03fc4ae4a960205)
---
 sys/kern/kern_switch.c | 28 +++++++++++++++++++---------
 1 file changed, 19 insertions(+), 9 deletions(-)

diff --git a/sys/kern/kern_switch.c b/sys/kern/kern_switch.c
index 42fee61a99a0..c6f749984217 100644
--- a/sys/kern/kern_switch.c
+++ b/sys/kern/kern_switch.c
@@ -215,12 +215,6 @@ critical_exit_preempt(void)
 	struct thread *td;
 	int flags;
 
-	/*
-	 * If td_critnest is 0, it is possible that we are going to get
-	 * preempted again before reaching the code below. This happens
-	 * rarely and is harmless. However, this means td_owepreempt may
-	 * now be unset.
-	 */
 	td = curthread;
 	if (td->td_critnest != 0)
 		return;
@@ -228,13 +222,29 @@ critical_exit_preempt(void)
 		return;
 
 	/*
-	 * Microoptimization: we committed to switch,
-	 * disable preemption in interrupt handlers
-	 * while spinning for the thread lock.
+	 * Temporarily disable preemption, as thread_lock() can enter/exit
+	 * spinlock sections multiple times if the thread lock can't be acquired
+	 * right away or changes.  A call to spinlock_exit() there causes
+	 * a fast-growing stack of recursive calls with critical_exit_preempt()
+	 * as long as the condition persists (provided 'td_owepreempt' is still
+	 * non-zero), easily leading to stack exhaustion.
 	 */
 	td->td_critnest = 1;
 	thread_lock(td);
 	td->td_critnest--;
+	/*
+	 * We might have been preempted after the test on 'td_critnest' above
+	 * and before returning from the thread_lock() call, on reception of an
+	 * IPI_PREEMPT or some interrupt whose handler uses critical sections.
+	 * If that is the case, 'td_owepreempt' was reset and there is no need
+	 * to switch to another thread.  We are in a spinlock section here so
+	 * 'td_owepreempt' is stable and no incoming preemption will be lost.
+	 */
+	if (__predict_false(td->td_owepreempt == 0)) {
+		thread_unlock(td);
+		return;
+	}
+
 	flags = SW_INVOL | SW_PREEMPT;
 	if (TD_IS_IDLETHREAD(td))
 		flags |= SWT_IDLE;