git: fd6a3eae051e - main - arm64 pmap: Invalidate the TLB in pmap_enter_largepage()

From: Alan Cox <alc_at_FreeBSD.org>
Date: Tue, 17 Nov 2026 16:39:39 UTC
The branch main has been updated by alc:

URL: https://cgit.FreeBSD.org/src/commit/?id=fd6a3eae051ee553403c016c9a93d57e680d5f7a

commit fd6a3eae051ee553403c016c9a93d57e680d5f7a
Author:     Alan Cox <alc@FreeBSD.org>
AuthorDate: 2026-11-13 09:14:52 +0000
Commit:     Alan Cox <alc@FreeBSD.org>
CommitDate: 2026-11-17 16:20:18 +0000

    arm64 pmap: Invalidate the TLB in pmap_enter_largepage()
    
    The amd64 version of pmap_enter_largepage(), on which the arm64 version
    was based, never invalidates the TLB.  That is correct on amd64, because
    a page fault automatically invalidates the TLB entry for the faulting
    address.  However, on arm64 an entry that causes a permission fault may
    remain in the TLB.  Consequently, we need to invalidate the TLB whenever
    pmap_enter_largepage() changes a valid mapping, unless only the wired
    bit changes.
    
    This only impacts large page shared memory objects created by
    shm_create_largepage() or memfd_create() with MFD_HUGETLB.
    
    Fixes:  4168aedcde0b ("Add largepage support to the arm64 pmap.")
    Reviewed by:    kib, markj
    MFC after:      1 week
    Differential Revision:  https://reviews.freebsd.org/D61549
---
 sys/arm64/arm64/pmap.c | 20 +++++++++++++++++++-
 1 file changed, 19 insertions(+), 1 deletion(-)

diff --git a/sys/arm64/arm64/pmap.c b/sys/arm64/arm64/pmap.c
index c7fa75d1f25f..c4435d7dd3eb 100644
--- a/sys/arm64/arm64/pmap.c
+++ b/sys/arm64/arm64/pmap.c
@@ -5645,8 +5645,26 @@ restart:
 			pmap_store(tl3p, newpte);
 			newpte += L3_SIZE;
 		}
+		/*
+		 * Revert to the last L3 entry within the L3C superpage to
+		 * detect wiring-only changes.
+		 */
+		newpte -= L3_SIZE;
 	}
-	dsb(ishst);
+
+	/*
+	 * A page fault does not automatically invalidate the TLB entry for a
+	 * valid mapping, so we must perform an invalidation on any change to
+	 * a valid mapping, including changes that add permissions.  The only
+	 * exception is a wiring-only change.  One TLB invalidation suffices
+	 * for L1 or L2 block mappings.
+	 */
+	if ((origpte & ATTR_DESCR_VALID) != 0 &&
+	    ((origpte ^ newpte) & ~ATTR_SW_WIRED) != 0)
+		pmap_invalidate_range(pmap, va, va + (psind == 1 ?
+		    L3C_SIZE : PAGE_SIZE), true);
+	else
+		dsb(ishst);
 
 	if ((origpte & ATTR_DESCR_VALID) == 0)
 		pmap_resident_count_inc(pmap, pagesizes[psind] / PAGE_SIZE);