From nobody Tue Oct 20 16:47:20 2026 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4j8JHF0pr4z6vm9W for ; Tue, 20 Oct 2026 16:47:21 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4j8JHF01hKz4YXL for ; Tue, 20 Oct 2026 16:47:21 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1792514841; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=D7WL/jPFfVRrc9yEtq7m6BlgZEc35EmIBSXuRfwMzOo=; b=i18e0tmFsgnfdZjfyPXhnb0wBpm3RUoT4usuXd4CMZafHj17Z54OgVSddren3fIARDFb0K st8gndRii6hF5Emb6IGMuw+jOtNgoXdzrOVuYK8pY4IFKPS4sxlTWranWb3BPpjqJ+iH6e ehQPvBB7nGvFpeWZEeisK7mXdZuNEXZXTyDc+H5hfznsVlBuCsXRUbE7fyEOSr1xHYfAUB YfyMGSsI7Vq6fvjNpDpw4z0QBP8rY7NH3TXpiPn/HB4S3vlEtg5NLaKXFKSt7ANAgCWDJL e6V1jWwokvDTM9UYiE1KDnldTC3GTlJKLZNlJ40CYfZIQwTOVocK3Yp30rkXLw== ARC-Seal: i=1; a=rsa-sha256; d=freebsd.org; s=dkim; cv=none; t=1792514841; b=VCKwiA3B4w7R2yDsJ4DQBKH1vQkrQYT1sa74cxKvDKMX3BoHbnGI6diFvqgmvgrLnbwYVi a3CFjBEYfkoB/I1XB3JsySNJJ9uxwIWrxBPGYUzhzNJgxZTfL0Kdf/1oURrJQP3SgyxwW/ qpUPTGFaXPV93Rhu3P/edIKr66u7pXJQ59cmBGfs/lxzfSBVdjQjujfhbKM1+2iYmRyXnG 6lPcdzSpwdsNlqt0nxZ8/cNc33+nXyOma0jodU0Phll+58WZ2BzrpEBKxm6SpVQ1lNfqCq Af3FvP8tjhh+ZUO1kJU+kwtH6Q8aWAu3+jwO2s6HORnhOW3hK4MYzt7cv3/mKQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1792514841; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=D7WL/jPFfVRrc9yEtq7m6BlgZEc35EmIBSXuRfwMzOo=; b=uKcqYHEYlTIwNiGJmiInMcOWORZi4QcP1zYlyaItaLETxfXvjaBSI6CkSg6jQyR61SyaN/ aNzB6gG6BpEAN7shR3znDk7cYHwJGCd7c4qQZALokGReqh2VYEtp+YP8WZJGrY7RxGJT9l daL3IrVvO627ybMCqZ12cyF8bHwP64mCWZ8dIf3hf/LB3anrGZlpG9ZoJz8P9MSIenILQU RE6Y4MIqKyQ8rJYD2SKcbIhniHNZxik+hKrKbqwRtLNDvgHYC91qfsKOdRK72aHh+ChgIz Oj8saV05Phxdxmqo+7w7wxquT/pRRr6P1zshPXkSRAbhS593DShGc9tlzWwhSA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4j8JHD6WRhzFLp for ; Tue, 20 Oct 2026 16:47:20 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 2251f by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Tue, 20 Oct 2026 16:47:20 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Olivier Certner Subject: git: b83bd6fec075 - main - x86: ipi_bitmap_handler(): Preempt last, reduce critical section List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org List-Id: Commit messages for the main branch of the src repository List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: olce X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: b83bd6fec075f4acf614283cd66a691111847881 Auto-Submitted: auto-generated Date: Tue, 20 Oct 2026 16:47:20 +0000 Message-Id: <6ad79b18.2251f.74feec46@gitrepo.freebsd.org> The branch main has been updated by olce: URL: https://cgit.FreeBSD.org/src/commit/?id=b83bd6fec075f4acf614283cd66a691111847881 commit b83bd6fec075f4acf614283cd66a691111847881 Author: Olivier Certner AuthorDate: 2026-10-13 14:32:15 +0000 Commit: Olivier Certner CommitDate: 2026-10-20 16:46:07 +0000 x86: ipi_bitmap_handler(): Preempt last, reduce critical section Preemption should happen last so that other IPIs have a chance to be executed in due course, in particular hardclockintr(). So, move handling of IPI_PREEMPT at the end. Reduce the critical section to be in force only around hardclockintr(), which needs it to execute entirely without preemption (because of code it calls that can enter a critical section; interrupts are already disabled in this handler, as usual). The critical_exit() following hardclockintr() may cause preemption if due. If that happens, there is no need to preempt again on IPI_PREEMPT, so make sure sched_preempt() is not called in this case. This fixes inadvertent removal of the optimization that sched_preempt() should be called outside of a critical section to perform an immediate context switch in the case of IPI_HARDCLOCK and IPI_PREEMPT being handled at the same time. While here, move IPI_AST handling just before IPI_PREEMPT. We currently do not need to do anything for it, but if we were to, it is likely that would have to happen after other IPIs but IPI_PREEMPT. Reviewed by: kib Fixes: ff326a187907 ("x86: Restore the critical section around ...") MFC after: 1 week Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D60861 --- sys/x86/x86/mp_x86.c | 45 +++++++++++++++++++++++++++------------------ 1 file changed, 27 insertions(+), 18 deletions(-) diff --git a/sys/x86/x86/mp_x86.c b/sys/x86/x86/mp_x86.c index eff18b1875af..8fdeff859561 100644 --- a/sys/x86/x86/mp_x86.c +++ b/sys/x86/x86/mp_x86.c @@ -1335,6 +1335,7 @@ ipi_bitmap_handler(struct trapframe *frame) struct thread *td; int cpu = PCPU_GET(cpuid); u_int ipi_bitmap; + bool do_preempt = true; kasan_mark(frame, sizeof(*frame), sizeof(*frame), 0); kmsan_mark(frame, sizeof(*frame), KMSAN_STATE_INITED); @@ -1343,16 +1344,6 @@ ipi_bitmap_handler(struct trapframe *frame) ipi_bitmap = atomic_readandclear_int(&cpuid_to_pcpu[cpu]-> pc_ipi_bitmap); - /* - * sched_preempt() must be called to clear the pending preempt - * IPI to enable delivery of further preempts. However, the - * critical section will cause extra scheduler lock thrashing - * when used unconditionally. Only critical_enter() if - * hardclock must also run, which requires the section entry. - */ - if (ipi_bitmap & (1 << IPI_HARDCLOCK)) - critical_enter(); - td->td_intr_nesting_level++; oldframe = td->td_intr_frame; td->td_intr_frame = frame; @@ -1360,11 +1351,24 @@ ipi_bitmap_handler(struct trapframe *frame) if (ipi_bitmap & (1 << IPI_TRACE)) stack_capture_intr(); #endif - if (ipi_bitmap & (1 << IPI_PREEMPT)) { + if (ipi_bitmap & (1 << IPI_HARDCLOCK)) { #ifdef COUNT_IPIS - (*ipi_preempt_counts[cpu])++; + (*ipi_hardclock_counts[cpu])++; #endif - sched_preempt(td); + /* + * Enter a critical section so that hardclockintr() can execute + * entirely without preemption (as it calls code that can enter + * critical sections). + */ + critical_enter(); + hardclockintr(); + /* + * If we are going to preempt now, do not let sched_preempt() do + * an unnecessary extra preemption below. + */ + if (td->td_owepreempt != 0) + do_preempt = false; + critical_exit(); } if (ipi_bitmap & (1 << IPI_AST)) { #ifdef COUNT_IPIS @@ -1372,16 +1376,21 @@ ipi_bitmap_handler(struct trapframe *frame) #endif /* Nothing to do for AST */ } - if (ipi_bitmap & (1 << IPI_HARDCLOCK)) { + /* + * sched_preempt() causes an immediate switch to another thread when not + * executing in a critical section. In order to ensure that all other + * IPIs are processed in due course (in particular, hardclockintr()), it + * has to be called last. + */ + if (ipi_bitmap & (1 << IPI_PREEMPT)) { #ifdef COUNT_IPIS - (*ipi_hardclock_counts[cpu])++; + (*ipi_preempt_counts[cpu])++; #endif - hardclockintr(); + if (do_preempt) + sched_preempt(td); } td->td_intr_frame = oldframe; td->td_intr_nesting_level--; - if (ipi_bitmap & (1 << IPI_HARDCLOCK)) - critical_exit(); } /*