From nobody Tue Sep 22 15:41:50 2026 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4hq48b2wxjz6sNbG for ; Tue, 22 Sep 2026 15:41:51 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "YR2" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4hq48Z5dyqz4vFD for ; Tue, 22 Sep 2026 15:41:50 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1790091710; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=l5JfSnK2no7HOwCa2aP1rS7KEJ8uD7Yj7QZ68YPmLLs=; b=c0TClmxssgL3YopBZfVKVGroek8V47+ZfrSrmfI3fd3uzptxRTTwLOsKFhnw1GO0gcjD0G Ebas5027ONsLEWTo2HsxlmwZV/jDo3sZQ3Z9a5Nak7JdzdnLaa/mtAmv5mcF8TdM6U3oFF ej1z3OGK2RSa/xgrvMwGMahgGNSP/ZGSoTtFgUFfnxfS4jU2nSyHNOiNsqXVW1dlqw0+1X tgzGD8cU0g52pE98MGjT3E6CIVRevGnAznKLjyfIxa6UGUisN+Ozwhk9F8FTUPEVchJp1l Zx4EBNgS+b5zlQisewkv0VQqKbQzy+3GIbez2HOFRhmnVPw930XEqg23MO/++w== ARC-Seal: i=1; a=rsa-sha256; d=freebsd.org; s=dkim; cv=none; t=1790091710; b=Y1WljGNdJ5aqqM+UIkNw/NUCqViaXJgqq8xyTYaOYyaDd/Bl5bHAmYvHUWgyvQ8bz8m8rg O5+b8TqO0KvCO1PN+bpNpbjKdM+kuRW50vIOE+jBc0ibpe67G2ESqGoIixAuS6DXJy54ap +mQ6P3WGGQMNEWIYOfJwhHw+OlEEx/LtBcJD+P/vKd6X1wPfjPLNUftDvndnN6dXHJiOeW i0PuiU3fbLvTaFBeQH68OL5R+y+Xl9aNmWfs37OYX4L8fFv3Ljcoom73EkWqyeExS6TZZ2 aEikX5lIT9AQmpI8GBbXob/wgqCHUsu+rt2oCQ9mXWCNTd5s97/4rfiqVy5aGQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1790091710; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=l5JfSnK2no7HOwCa2aP1rS7KEJ8uD7Yj7QZ68YPmLLs=; b=vPdWZfPhVFeQp8LYbQt+GUJTxEe/ZsE1nts9KFUWgGT7ipzfxBTi0WaCNx9dW8kd5JInkR oJm0AarrjBIsFIgVzl0yUEk661447v/GomRpZdpaKHA1EKfyKxJqP6X59vhe/4YeUOtKms eG5c2yxWfM8tY+gOhfHNtDhWq9U3pIa+cTTZPhwNmx5APYefw5bfeB6kvzG57svKvhhCQ4 y9C7/EPfy1fnxTONwht/get0iN+5vZYg86XuyQ39CFzIdKL4aS8/u55XcmO9FX2U5qy1qB SU9u+DQ4ZidNay/L3cOD58xAEZFFNbmIygAxPwtLkudn5fO9cvV0toWD8IJDBw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4hq48Z4F3tz1CXM for ; Tue, 22 Sep 2026 15:41:50 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 348c6 by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Tue, 22 Sep 2026 15:41:50 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Konstantin Belousov Subject: git: e318f24c0f53 - main - amd64: support for tracking per-thread 'disable splitlocks' state List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kib X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: e318f24c0f53b52bd280b827509ad752db0497bf Auto-Submitted: auto-generated Date: Tue, 22 Sep 2026 15:41:50 +0000 Message-Id: <6ab2a1be.348c6.3fe519ef@gitrepo.freebsd.org> The branch main has been updated by kib: URL: https://cgit.FreeBSD.org/src/commit/?id=e318f24c0f53b52bd280b827509ad752db0497bf commit e318f24c0f53b52bd280b827509ad752db0497bf Author: Konstantin Belousov AuthorDate: 2026-09-18 14:33:54 +0000 Commit: Konstantin Belousov CommitDate: 2026-09-22 15:40:40 +0000 amd64: support for tracking per-thread 'disable splitlocks' state If hw supports it, on atomic operation that requires exclusive ownership of more than one cache line, #AC is generated. The state is maintained as the arch-private TDF_MD_SPLITLOCK_AC flag. The state is inherited on thread creation from the thread spawning the new one. It is cleared on exec. Reviewed by: markj Sponsored by: The FreeBSD Foundation MFC after: 1 week Differential revision: https://reviews.freebsd.org/D59815 --- sys/amd64/amd64/exec_machdep.c | 10 ++++++++++ sys/amd64/amd64/machdep.c | 36 ++++++++++++++++++++++++++++++++++++ sys/amd64/amd64/pmap.c | 14 ++++++++++++++ sys/amd64/amd64/vm_machdep.c | 2 ++ sys/amd64/ia32/ia32_signal.c | 1 + sys/amd64/include/md_var.h | 5 +++++ sys/amd64/include/proc.h | 2 ++ sys/amd64/linux/linux_sysvec.c | 1 + sys/amd64/linux32/linux32_sysvec.c | 1 + 9 files changed, 72 insertions(+) diff --git a/sys/amd64/amd64/exec_machdep.c b/sys/amd64/amd64/exec_machdep.c index d5d3342b7dff..19b087c783d3 100644 --- a/sys/amd64/amd64/exec_machdep.c +++ b/sys/amd64/amd64/exec_machdep.c @@ -406,6 +406,15 @@ x86_clear_dbregs(struct pcb *pcb) clear_pcb_flags(pcb, PCB_DBREGS); } +void +exec_splitlock(struct thread *td) +{ + if (ia32_splitlock_force) + enable_splitlock(td); + else if (ia32_splitlock) + disable_splitlock(td); +} + /* * Reset registers to default values on exec. */ @@ -443,6 +452,7 @@ exec_setregs(struct thread *td, struct image_params *imgp, uintptr_t stack) regs->tf_flags = TF_HASSEGS; x86_clear_dbregs(pcb); + exec_splitlock(td); /* * Drop the FP state if we hold it, so that the process gets a diff --git a/sys/amd64/amd64/machdep.c b/sys/amd64/amd64/machdep.c index c34dd7818b8c..b10c58955ce0 100644 --- a/sys/amd64/amd64/machdep.c +++ b/sys/amd64/amd64/machdep.c @@ -1928,6 +1928,42 @@ safe_read(vm_offset_t addr, char *valp) return (uiomove_mem(UIO_MEM_KMEM, &uio)); } +void +enable_splitlock_ac(void) +{ + MPASS(ia32_splitlock); + wrmsr(MSR_MEMORY_CTL, rdmsr(MSR_MEMORY_CTL) | + MSR_MEMORY_CTL_SPLITLOCK); +} + +void +enable_splitlock(struct thread *td) +{ + MPASS(td == curthread); + td->td_md.md_td_flags |= TDF_MD_SPLITLOCK_AC; + critical_enter(); + enable_splitlock_ac(); + critical_exit(); +} + +void +disable_splitlock_ac(void) +{ + MPASS(ia32_splitlock); + wrmsr(MSR_MEMORY_CTL, rdmsr(MSR_MEMORY_CTL) & + ~MSR_MEMORY_CTL_SPLITLOCK); +} + +void +disable_splitlock(struct thread *td) +{ + MPASS(td == curthread); + td->td_md.md_td_flags &= ~TDF_MD_SPLITLOCK_AC; + critical_enter(); + disable_splitlock_ac(); + critical_exit(); +} + #ifdef KDB /* diff --git a/sys/amd64/amd64/pmap.c b/sys/amd64/amd64/pmap.c index 8f8d91c768b7..74c1fc0d755e 100644 --- a/sys/amd64/amd64/pmap.c +++ b/sys/amd64/amd64/pmap.c @@ -10236,8 +10236,22 @@ DEFINE_IFUNC(static, void, pmap_activate_sw_mode, (struct thread *, pmap_t, void pmap_activate_sw(struct thread *td) { + struct thread *oldtd; pmap_t oldpmap, pmap; u_int cpuid; + bool oldtd_sl, td_sl; + + oldtd = curthread; + if (ia32_splitlock && oldtd != td) { + oldtd_sl = (atomic_load_int(&oldtd->td_md.md_td_flags) & + TDF_MD_SPLITLOCK_AC) != 0; + td_sl = (atomic_load_int(&td->td_md.md_td_flags) & + TDF_MD_SPLITLOCK_AC) != 0; + if (oldtd_sl && !td_sl) + disable_splitlock_ac(); + else if (!oldtd_sl && td_sl) + enable_splitlock_ac(); + } oldpmap = PCPU_GET(curpmap); pmap = vmspace_pmap(td->td_proc->p_vmspace); diff --git a/sys/amd64/amd64/vm_machdep.c b/sys/amd64/amd64/vm_machdep.c index 58af1cb8665c..49d46b31233d 100644 --- a/sys/amd64/amd64/vm_machdep.c +++ b/sys/amd64/amd64/vm_machdep.c @@ -182,6 +182,8 @@ copy_thread(struct thread *td1, struct thread *td2) /* Setup to release spin count in fork_exit(). */ td2->td_md.md_spinlock_count = 1; td2->td_md.md_saved_flags = PSL_KERNEL | PSL_I; + td2->td_md.md_td_flags = td1->td_md.md_td_flags & + TDF_MD_SPLITLOCK_AC; pmap_thread_init_invl_gen(td2); /* diff --git a/sys/amd64/ia32/ia32_signal.c b/sys/amd64/ia32/ia32_signal.c index 3b26244932b4..84912b292861 100644 --- a/sys/amd64/ia32/ia32_signal.c +++ b/sys/amd64/ia32/ia32_signal.c @@ -981,6 +981,7 @@ ia32_setregs(struct thread *td, struct image_params *imgp, uintptr_t stack) regs->tf_flags = TF_HASSEGS; x86_clear_dbregs(pcb); + exec_splitlock(td); fpstate_drop(td); diff --git a/sys/amd64/include/md_var.h b/sys/amd64/include/md_var.h index a46ae8dfe794..b85e442e559a 100644 --- a/sys/amd64/include/md_var.h +++ b/sys/amd64/include/md_var.h @@ -105,6 +105,11 @@ void get_fpcontext(struct thread *td, struct __mcontext *mcp, char **xfpusave, size_t *xfpusave_len); int set_fpcontext(struct thread *td, struct __mcontext *mcp, char *xfpustate, size_t xfpustate_len); +void enable_splitlock_ac(void); +void enable_splitlock(struct thread *td); +void disable_splitlock_ac(void); +void disable_splitlock(struct thread *td); +void exec_splitlock(struct thread *td); void wrmsr_early_safe_start(void); void wrmsr_early_safe_end(void); diff --git a/sys/amd64/include/proc.h b/sys/amd64/include/proc.h index 9ecc5da15749..ce96fa0fedff 100644 --- a/sys/amd64/include/proc.h +++ b/sys/amd64/include/proc.h @@ -80,6 +80,8 @@ struct mdthread { void *md_usr_fpu_save; }; +#define TDF_MD_SPLITLOCK_AC 0x00000001 /* Disable split cache lines */ + struct mdproc { struct proc_ldt *md_ldt; /* (t) per-process ldt */ struct system_segment_descriptor md_ldt_sd; diff --git a/sys/amd64/linux/linux_sysvec.c b/sys/amd64/linux/linux_sysvec.c index ecb497c61a1a..1134efd1c726 100644 --- a/sys/amd64/linux/linux_sysvec.c +++ b/sys/amd64/linux/linux_sysvec.c @@ -266,6 +266,7 @@ linux_exec_setregs(struct thread *td, struct image_params *imgp, regs->tf_flags = TF_HASSEGS; x86_clear_dbregs(pcb); + exec_splitlock(td); /* * Drop the FP state if we hold it, so that the process gets a diff --git a/sys/amd64/linux32/linux32_sysvec.c b/sys/amd64/linux32/linux32_sysvec.c index c4de547b2bf5..f42654967e8f 100644 --- a/sys/amd64/linux32/linux32_sysvec.c +++ b/sys/amd64/linux32/linux32_sysvec.c @@ -608,6 +608,7 @@ linux_exec_setregs(struct thread *td, struct image_params *imgp, regs->tf_rbx = (register_t)imgp->ps_strings; x86_clear_dbregs(pcb); + exec_splitlock(td); fpstate_drop(td);