From nobody Mon Aug 10 16:01:44 2026 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4hJfdP13N7z6p08d for ; Mon, 10 Aug 2026 16:01:45 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "YR1" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4hJfdN6hWNz3nCR for ; Mon, 10 Aug 2026 16:01:44 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1786377705; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=cdaPnn8rpn4xuKPS7QctALBY3QjW/9nA6AG60IEYlAY=; b=isk+YdUt0Rz6xk0NKca2VMfL2o60X525Fma1e+0CuA9SPpQ01H6VJw7Zp/v6nC+aybkWEU vw3fmtt4RwnSLGSpaXQxkHKBOWD/JDO3Yga0YpoA8UrwV4+8OvSZtyEpdGbv7J7b0dXIRB gZqCIcJGT1y0eBeBT/gkDkoFveUIx+2eOWNGZY2R+LWTjVCJzA2ElTKl4My6v9lIFfsKd7 HQgMucbNOU0z2F8CyIYq6qQpldkP92/XhX1gcLVPTHKApEg0ciT9cOk8U3HF1zmWO4RYl7 EOsEhzXKpGhQKe0CPbPJnkrtEVcUX2uT6610SFKuLKlL2nCZqmj4cexOxujJhA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1786377705; a=rsa-sha256; cv=none; b=Fz0GNZa09oN3tpDCcU8ABk7/qohqoLRPjm535Pt0pJ2YDdKRRbvugeRh+xrpLaDbFaC6mc XofNovk3Rs+49kN+xybaF79auK1PMIwi568l/txtg0TJmE/wMDvA4rRNh0evkDufoBSd1F Te7yvS1r5KGqXDR79mTa8IZmA3MVCR9mJmVRNcmK9391Xuzyd4vesdClPiDgKvEKLmGbvT w0x8Fwp1x0S1nKID4BOXfqItD3xqHG9v5U4fPI4xODt5EXVlHJvQ7khNSGUoiq7KBBso+T I+2YuuucJ+2uvqxFCS9f9MPOAc4Xwu5BYJ52GNLJCU2UzHikFvv74DSafKoXUA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1786377705; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=cdaPnn8rpn4xuKPS7QctALBY3QjW/9nA6AG60IEYlAY=; b=sE589TClUdBGoP7FkT4Uh6uzhe9llW6ouz2Eo3TMqD30+9jakPxYTuicyWxdiQqu3aN9se AH9zQ8eXAM4mIvRmmRqm5Yl2l4DO7W28zBk+i3DnADn4Bl47pDwaxREuQxQ2siMct7vHA7 0jin8+R0Qre69z+g359z4XB92r8DMAr5JlOvDnjggbAabmHu9a9KVIg8YZUyYsWVpj366n 0J10EZppJ9OK8sQTmJWrQINT1h+8242o+ujUnwHlq4D8gZDPdSYSRc+xHoYF+aTjLudgBX w+PUuDh1H449K3dXx2ulqh07WPiHipPxAjJmhekV8mZ42GvjTX/eSrFVJnX2WQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4hJfdN5fwbz1PR for ; Mon, 10 Aug 2026 16:01:44 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 37105 by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Mon, 10 Aug 2026 16:01:44 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Bojan Novk=?utf-8?Q?ovi=C4=87?= Subject: git: 99360212c739 - main - riscv/pmap.c: Add an Svinval-aware variant of pmap_invalidate_range List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: bnovkov X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 99360212c7392f2b0b3932b40f005c71e943b65f Auto-Submitted: auto-generated Date: Mon, 10 Aug 2026 16:01:44 +0000 Message-Id: <6a79f5e8.37105.d66886@gitrepo.freebsd.org> The branch main has been updated by bnovkov: URL: https://cgit.FreeBSD.org/src/commit/?id=99360212c7392f2b0b3932b40f005c71e943b65f commit 99360212c7392f2b0b3932b40f005c71e943b65f Author: Bojan Novković AuthorDate: 2026-06-16 14:55:48 +0000 Commit: Bojan Novković CommitDate: 2026-08-10 16:01:28 +0000 riscv/pmap.c: Add an Svinval-aware variant of pmap_invalidate_range This change introduces a variant of `pmap_invalidate_range` that uses the fine-grained TLB invalidation instructions introduced by the Svinval extension. These instructions allow for more efficient TLB flushing on certain implementations. Under this new scheme, `pmap_invalidate_range` was converted to an ifunc that selects the appropriate variant during boot. Event: BSDCan 2026 Reviewed by: markj, mhorne Differential Revision: https://reviews.freebsd.org/D57624 --- sys/riscv/riscv/pmap.c | 73 ++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 71 insertions(+), 2 deletions(-) diff --git a/sys/riscv/riscv/pmap.c b/sys/riscv/riscv/pmap.c index 67123f9bfcc8..b3f03eeaab9f 100644 --- a/sys/riscv/riscv/pmap.c +++ b/sys/riscv/riscv/pmap.c @@ -159,6 +159,7 @@ #include #include +#include #include #include #include @@ -345,6 +346,7 @@ static int pmap_unuse_pt(pmap_t, vm_offset_t, pd_entry_t, struct spglist *); static int pmap_change_attr_locked(void *va, vm_size_t size, int mode); static uint64_t pmap_satp_mode(void); +static void pmap_invalidate_all(pmap_t pmap); #define pmap_clear(pte) pmap_store(pte, 0) #define pmap_clear_bits(pte, bits) atomic_clear_64(pte, bits) @@ -1077,7 +1079,7 @@ pmap_init(void) * sfence_vma() on remote CPUs. */ static void -pmap_invalidate_page(pmap_t pmap, vm_offset_t va) +pmap_invalidate_page_sbi(pmap_t pmap, vm_offset_t va) { cpuset_t mask; @@ -1092,7 +1094,7 @@ pmap_invalidate_page(pmap_t pmap, vm_offset_t va) } static void -pmap_invalidate_range(pmap_t pmap, vm_offset_t sva, vm_offset_t eva) +pmap_invalidate_range_sbi(pmap_t pmap, vm_offset_t sva, vm_offset_t eva) { cpuset_t mask; @@ -1111,6 +1113,73 @@ pmap_invalidate_range(pmap_t pmap, vm_offset_t sva, vm_offset_t eva) sched_unpin(); } +#define PMAP_SVINVAL_THRESHOLD (2 * L2_SIZE) + +struct svinval_args { + vm_offset_t sva; + vm_offset_t eva; +}; + +static void +pmap_invalidate_range_svinval_cb(void *arg) +{ + struct svinval_args *args = arg; + vm_offset_t va; + + sfence_w_inval(); + for (va = args->sva; va < args->eva; va += PAGE_SIZE) + sinval_vma_page(va); + sfence_inval_ir(); +} + +static void +pmap_invalidate_range_svinval(pmap_t pmap, vm_offset_t sva, vm_offset_t eva) +{ + struct svinval_args args; + + if (CPU_EMPTY(&pmap->pm_active)) + return; + + if (eva - sva >= PMAP_SVINVAL_THRESHOLD) { + pmap_invalidate_all(pmap); + return; + } + + sched_pin(); + args.sva = sva; + args.eva = eva; + fence(); + if (smp_started) + smp_rendezvous_cpus(pmap->pm_active, smp_no_rendezvous_barrier, + pmap_invalidate_range_svinval_cb, + smp_no_rendezvous_barrier, &args); + else + pmap_invalidate_range_svinval_cb(&args); + sched_unpin(); +} + +static void +pmap_invalidate_page_svinval(pmap_t pmap, vm_offset_t va) +{ + pmap_invalidate_range_svinval(pmap, va, va + PAGE_SIZE); +} + +DEFINE_IFUNC(, void, pmap_invalidate_range, + (pmap_t pmap, vm_offset_t sva, vm_offset_t eva)) +{ + if (has_svinval) + return (pmap_invalidate_range_svinval); + return (pmap_invalidate_range_sbi); +} + +DEFINE_IFUNC(, void, pmap_invalidate_page, + (pmap_t pmap, vm_offset_t va)) +{ + if (has_svinval) + return (pmap_invalidate_page_svinval); + return (pmap_invalidate_page_sbi); +} + static void pmap_invalidate_all(pmap_t pmap) {