From nobody Thu Mar 05 15:14:21 2026 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4fRY3d62Wsz6TBSW for ; Thu, 05 Mar 2026 15:14:21 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R12" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4fRY3d5GZnz3Znx for ; Thu, 05 Mar 2026 15:14:21 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1772723661; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=GTmNKIhzYkbUxyaCn9WrwCuhJrAvcSVygP706IIqj7w=; b=XvpfOobqfawjUgEwvaeFeMMS/r6A1eMK3cXWuzGAiNk6xUOcn7qb/en2U6HJAyK42Wv5ST wzzzmQhLW9P657HbLppvaIpftIrIFzMKLjaYOEgTCqilTxa7j5+SgkH4tRcpR3Djl7+e+v HIDVoBtZksol4P48wi78sRS7X08UxOW2RGY5xETb169MBp2bRshpU+y6Pq7pZnOUAi1xlA z9J5GnePN/mJz9xwaVu/u0BZ20zDY1+h215o+gC3amXu1m3pYkrZfVUHTsonBMXsjOoYtU hZHvAv0CBTA00NpybOv7egd9PGfaNUqEOtP9q68uSNWGKMSajPzyJ6jxAi2YxA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1772723661; a=rsa-sha256; cv=none; b=qrtIzkBaGzEah8LXXHU1+KGDwCSiKyBpC54597edoWtBkE8pHb5Fc3xirWan1zdDARAg1q qdU1TdgjDYm9969771V39YQ10uit2jL3ohYyENMr1ubnwtn8YK3UBQjJzDURCr04hKPT5q JdAVmhQfckJNbv98dzTXPFAxxahi076TVH+8Usype3B/pb7kCqYcHSHIoWN5Sltew4DTjL J1rZ4CKsyimDpHAOC4p2kDMztnqyC02TtZouHv6w1zmdHSiK0w4YMiPpwefolyrP4KxYz7 Jt9O6QDffjHdm1SJwbMM8dNsbMFe6uRWphSezIet2Mf2nnEg/SrH1znEvIeFPQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1772723661; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=GTmNKIhzYkbUxyaCn9WrwCuhJrAvcSVygP706IIqj7w=; b=oa/qx8viL6FXN98xsxiq4tagECgXH6Ul5gOTcAK6fB4pOWVHCKtC4veXFKqTJbYaMmAc0K KTCTCN8XzO6OV0j4urWRu2Pyi2nikChk4nU8lDhO9JwvHhcEoAj9+UKJSbi/uxOF9ilALi qhIbNgWPg2o2yp0bPZO+Tx5W6mFwd7hIjwsjqzk3RzwxoZAThHa3iG//4c/71bYdT9GqjV HJehW1+AKBqOAjF+QlLHmXZyz5TvHHmQlXeTa/YT013/bHWUQoD0iIVRuvozazK7FNjMRg Dwj131K4hWGIHLbTgSziMeBX7zVzgdJaHaPxrTweGyCflPqqtelba6B0S83nTA== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4fRY3d4qBpzjZk for ; Thu, 05 Mar 2026 15:14:21 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 30857 by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Thu, 05 Mar 2026 15:14:21 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Andrew Turner Subject: git: 80b4129bef8b - main - arm64: Optimise the repeated TLBI workaround List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: andrew X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 80b4129bef8b908eb19fe47853cb6e45e4513d76 Auto-Submitted: auto-generated Date: Thu, 05 Mar 2026 15:14:21 +0000 Message-Id: <69a99dcd.30857.41c1a2ad@gitrepo.freebsd.org> The branch main has been updated by andrew: URL: https://cgit.FreeBSD.org/src/commit/?id=80b4129bef8b908eb19fe47853cb6e45e4513d76 commit 80b4129bef8b908eb19fe47853cb6e45e4513d76 Author: Andrew Turner AuthorDate: 2026-03-05 14:28:01 +0000 Commit: Andrew Turner CommitDate: 2026-03-05 15:13:30 +0000 arm64: Optimise the repeated TLBI workaround It has been reported that the overhead of repeating all TLBI instructions is too large [1]. The Software Developer Errata Notices (SDEN) for the relevant Arm CPUs have been updated so a single "tlbi vale1is, xzr" followed by "dsb ish" is sufficient to work around the issues. Replace the places we repeat TLBI instructions with the new sequence. [1] https://lore.kernel.org/linux-arm-kernel/20260218164348.2022831-1-mark.rutland@arm.com/ Reviewed by: kib Sponsored by: Arm Ltd Differential Revision: https://reviews.freebsd.org/D55646 --- sys/arm64/arm64/pmap.c | 44 ++++++++++++++------------------------------ 1 file changed, 14 insertions(+), 30 deletions(-) diff --git a/sys/arm64/arm64/pmap.c b/sys/arm64/arm64/pmap.c index 25192bfef653..86ef7359bbe9 100644 --- a/sys/arm64/arm64/pmap.c +++ b/sys/arm64/arm64/pmap.c @@ -1926,17 +1926,13 @@ pmap_s1_invalidate_page(pmap_t pmap, vm_offset_t va, bool final_only) r = TLBI_VA(va); if (pmap == kernel_pmap) { pmap_s1_invalidate_kernel(r, final_only); - if (pmap_multiple_tlbi) { - dsb(ish); - pmap_s1_invalidate_kernel(r, final_only); - } } else { r |= ASID_TO_OPERAND(COOKIE_TO_ASID(pmap->pm_cookie)); pmap_s1_invalidate_user(r, final_only); - if (pmap_multiple_tlbi) { - dsb(ish); - pmap_s1_invalidate_user(r, final_only); - } + } + if (pmap_multiple_tlbi) { + dsb(ish); + __asm __volatile("tlbi vale1is, xzr" ::: "memory"); } dsb(ish); isb(); @@ -1978,24 +1974,16 @@ pmap_s1_invalidate_strided(pmap_t pmap, vm_offset_t sva, vm_offset_t eva, end = TLBI_VA(eva); for (r = start; r < end; r += TLBI_VA(stride)) pmap_s1_invalidate_kernel(r, final_only); - - if (pmap_multiple_tlbi) { - dsb(ish); - for (r = start; r < end; r += TLBI_VA(stride)) - pmap_s1_invalidate_kernel(r, final_only); - } } else { start = end = ASID_TO_OPERAND(COOKIE_TO_ASID(pmap->pm_cookie)); start |= TLBI_VA(sva); end |= TLBI_VA(eva); for (r = start; r < end; r += TLBI_VA(stride)) pmap_s1_invalidate_user(r, final_only); - - if (pmap_multiple_tlbi) { - dsb(ish); - for (r = start; r < end; r += TLBI_VA(stride)) - pmap_s1_invalidate_user(r, final_only); - } + } + if (pmap_multiple_tlbi) { + dsb(ish); + __asm __volatile("tlbi vale1is, xzr" ::: "memory"); } dsb(ish); isb(); @@ -2036,11 +2024,11 @@ pmap_s1_invalidate_all_kernel(void) { dsb(ishst); __asm __volatile("tlbi vmalle1is"); - dsb(ish); if (pmap_multiple_tlbi) { - __asm __volatile("tlbi vmalle1is"); dsb(ish); + __asm __volatile("tlbi vale1is, xzr" ::: "memory"); } + dsb(ish); isb(); } @@ -2058,17 +2046,13 @@ pmap_s1_invalidate_all(pmap_t pmap) dsb(ishst); if (pmap == kernel_pmap) { __asm __volatile("tlbi vmalle1is"); - if (pmap_multiple_tlbi) { - dsb(ish); - __asm __volatile("tlbi vmalle1is"); - } } else { r = ASID_TO_OPERAND(COOKIE_TO_ASID(pmap->pm_cookie)); __asm __volatile("tlbi aside1is, %0" : : "r" (r)); - if (pmap_multiple_tlbi) { - dsb(ish); - __asm __volatile("tlbi aside1is, %0" : : "r" (r)); - } + } + if (pmap_multiple_tlbi) { + dsb(ish); + __asm __volatile("tlbi vale1is, xzr" ::: "memory"); } dsb(ish); isb();