From nobody Wed Nov 04 13:25:15 2026 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4jKR580P0Tz6ymSW for ; Wed, 04 Nov 2026 13:25:16 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4jKR576GDQz3Lck for ; Wed, 04 Nov 2026 13:25:15 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1793798715; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=K6pX8VFYLwo++YnQXlYVqRtvWmBGe7VDNA512Z0/AzU=; b=or5l55iA6zdqz9GOhYnoXJ8itRBqatgA2+1tN4VJAxcbfwEs+I8vd7xbp6Vn71hLN63/Sq H/2erYLYjC18+xUSkx7FIr4xoCoBw0sfzUNB/CgVQJSeyegGbB/lYUrmPg4b5Fb3cOXQGi PKdXWrhS55X4YCb7jhieb2rcsVWdq59j7rFJgxZ7eeFQc7IqelKJcMY8s4j1yt8a4Ls83F 2Bug7kcQ5Y/A4UxnZTBf9vD+lF+x9hp6soYE5ab2tJu7Y+YVOnVekdeV7n9HqxMzLWAwGX Rf/LTt72O+3+42V0WDwmg5v4h3qAIJwdy574Iq3HWNJFD10VZP20r1TLAUg/rw== ARC-Seal: i=1; a=rsa-sha256; d=freebsd.org; s=dkim; cv=none; t=1793798715; b=tmawozHx+zVeN+xnyKtQT27JfI5LD6lu0rUGHoBky0sm4xDB8dcXp0a+syQs0nJTE5Ou/w f3qb5gUT/EONcfWHZPA4Cjk3onDaaL32TcHyYq8YANj++SuwOg7uVAxf3Jis+FonJ+iBXW OUPKouexv2hRJXsRHgI16g2MquUyOqsD5CWF/OGiCNel2ba9g5+230ExIye2bPuOW/eH/P Bzm2tSqNtYsCIjclQLLPUAKQQu2mXJCjCQXTaCjC2GXZJ2di6qAgHJTSzAtpZvPjDswK4f 2iFR8pciWiEbQtveZNSKxL3l49I3GUQueX0DtT8jqzuyHyopuNbc/DslX1j1cg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1793798715; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=K6pX8VFYLwo++YnQXlYVqRtvWmBGe7VDNA512Z0/AzU=; b=oD3HRF/2SQc6ySYmmC6E/kB1el23i+XUfM8a4t46naAa5w8Owarsxvi0ANFRrTAn3ZcwwR oystoj7NHxSa/8WgEetJus/Ga98S66N/3kw+jNfI/EoKb/yd21/sHgUBfnRc4Y/qhnFjbK NsT9RiqVxR/Q3v5XAeY8gVCIivx+l73fdRsFqd5kASpYNreK+AuhmbPSuX/DlHgzamTAvg Flb4MmC7WgLp85+8PPkLueY3E8A1oJvyr9oSrLEf61G/RlaP8U5Noi+XiA+shNZEBanc68 IcgoCBeQqOjtUgCvCOPTYl1asJiVssYdfKvKciYVJGEB9uulX6J7Z/U8wLBu8A== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4jKR575x9Bz13wb for ; Wed, 04 Nov 2026 13:25:15 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 3c7d2 by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Wed, 04 Nov 2026 13:25:15 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Olivier Certner Subject: git: d54cae6c5ad6 - stable/15 - x86: ipi_bitmap_handler(): Preempt last, reduce critical section List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org List-Id: Commit messages for all branches of the src repository List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: olce X-Git-Repository: src X-Git-Refname: refs/heads/stable/15 X-Git-Reftype: branch X-Git-Commit: d54cae6c5ad6b8096c2022bb6bcc57ab2acbd115 Auto-Submitted: auto-generated Date: Wed, 04 Nov 2026 13:25:15 +0000 Message-Id: <6aeb323b.3c7d2.2a3bfbb5@gitrepo.freebsd.org> The branch stable/15 has been updated by olce: URL: https://cgit.FreeBSD.org/src/commit/?id=d54cae6c5ad6b8096c2022bb6bcc57ab2acbd115 commit d54cae6c5ad6b8096c2022bb6bcc57ab2acbd115 Author: Olivier Certner AuthorDate: 2026-10-13 14:32:15 +0000 Commit: Olivier Certner CommitDate: 2026-11-04 13:21:37 +0000 x86: ipi_bitmap_handler(): Preempt last, reduce critical section Preemption should happen last so that other IPIs have a chance to be executed in due course, in particular hardclockintr(). So, move handling of IPI_PREEMPT at the end. Reduce the critical section to be in force only around hardclockintr(), which needs it to execute entirely without preemption (because of code it calls that can enter a critical section; interrupts are already disabled in this handler, as usual). The critical_exit() following hardclockintr() may cause preemption if due. If that happens, there is no need to preempt again on IPI_PREEMPT, so make sure sched_preempt() is not called in this case. This fixes inadvertent removal of the optimization that sched_preempt() should be called outside of a critical section to perform an immediate context switch in the case of IPI_HARDCLOCK and IPI_PREEMPT being handled at the same time. While here, move IPI_AST handling just before IPI_PREEMPT. We currently do not need to do anything for it, but if we were to, it is likely that would have to happen after other IPIs but IPI_PREEMPT. Reviewed by: kib Fixes: ff326a187907 ("x86: Restore the critical section around ...") MFC after: 1 week Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D60861 (cherry picked from commit b83bd6fec075f4acf614283cd66a691111847881) --- sys/x86/x86/mp_x86.c | 45 +++++++++++++++++++++++++++------------------ 1 file changed, 27 insertions(+), 18 deletions(-) diff --git a/sys/x86/x86/mp_x86.c b/sys/x86/x86/mp_x86.c index a01cc531d52e..2971c8077ce7 100644 --- a/sys/x86/x86/mp_x86.c +++ b/sys/x86/x86/mp_x86.c @@ -1335,6 +1335,7 @@ ipi_bitmap_handler(struct trapframe *frame) struct thread *td; int cpu = PCPU_GET(cpuid); u_int ipi_bitmap; + bool do_preempt = true; kasan_mark(frame, sizeof(*frame), sizeof(*frame), 0); kmsan_mark(frame, sizeof(*frame), KMSAN_STATE_INITED); @@ -1343,16 +1344,6 @@ ipi_bitmap_handler(struct trapframe *frame) ipi_bitmap = atomic_readandclear_int(&cpuid_to_pcpu[cpu]-> pc_ipi_bitmap); - /* - * sched_preempt() must be called to clear the pending preempt - * IPI to enable delivery of further preempts. However, the - * critical section will cause extra scheduler lock thrashing - * when used unconditionally. Only critical_enter() if - * hardclock must also run, which requires the section entry. - */ - if (ipi_bitmap & (1 << IPI_HARDCLOCK)) - critical_enter(); - td->td_intr_nesting_level++; oldframe = td->td_intr_frame; td->td_intr_frame = frame; @@ -1360,11 +1351,24 @@ ipi_bitmap_handler(struct trapframe *frame) if (ipi_bitmap & (1 << IPI_TRACE)) stack_capture_intr(); #endif - if (ipi_bitmap & (1 << IPI_PREEMPT)) { + if (ipi_bitmap & (1 << IPI_HARDCLOCK)) { #ifdef COUNT_IPIS - (*ipi_preempt_counts[cpu])++; + (*ipi_hardclock_counts[cpu])++; #endif - sched_preempt(td); + /* + * Enter a critical section so that hardclockintr() can execute + * entirely without preemption (as it calls code that can enter + * critical sections). + */ + critical_enter(); + hardclockintr(); + /* + * If we are going to preempt now, do not let sched_preempt() do + * an unnecessary extra preemption below. + */ + if (td->td_owepreempt != 0) + do_preempt = false; + critical_exit(); } if (ipi_bitmap & (1 << IPI_AST)) { #ifdef COUNT_IPIS @@ -1372,16 +1376,21 @@ ipi_bitmap_handler(struct trapframe *frame) #endif /* Nothing to do for AST */ } - if (ipi_bitmap & (1 << IPI_HARDCLOCK)) { + /* + * sched_preempt() causes an immediate switch to another thread when not + * executing in a critical section. In order to ensure that all other + * IPIs are processed in due course (in particular, hardclockintr()), it + * has to be called last. + */ + if (ipi_bitmap & (1 << IPI_PREEMPT)) { #ifdef COUNT_IPIS - (*ipi_hardclock_counts[cpu])++; + (*ipi_preempt_counts[cpu])++; #endif - hardclockintr(); + if (do_preempt) + sched_preempt(td); } td->td_intr_frame = oldframe; td->td_intr_nesting_level--; - if (ipi_bitmap & (1 << IPI_HARDCLOCK)) - critical_exit(); } /*