git: 1574ca1955f5 - main - powerpc/radix: take the pmap lock in mmu_radix_sync_icache()

The branch main has been updated by pkubaj:

URL: https://cgit.FreeBSD.org/src/commit/?id=1574ca1955f55151c4c76b978c8a772ac3abfa9f

commit 1574ca1955f55151c4c76b978c8a772ac3abfa9f
Author:     Piotr Kubaj <pkubaj@FreeBSD.org>
AuthorDate: 2026-09-01 06:52:06 +0000
Commit:     Piotr Kubaj <pkubaj@FreeBSD.org>
CommitDate: 2026-09-01 19:05:09 +0000

    powerpc/radix: take the pmap lock in mmu_radix_sync_icache()
    
    mmu_radix_sync_icache() walked the page tables with an unlocked
    pmap_extract() and passed the result straight to PHYS_TO_DMAP(),
    checking only that it was non-zero.  Nothing keeps the mapping - or the
    page table page holding it - alive across that window: if another thread
    of the same process tears a mapping down concurrently, the page table
    page can be freed and reused, so pmap_extract() reads arbitrary memory
    and returns a bogus physical address.  __syncicache() then dereferences
    an unmapped direct map address and the kernel takes a data storage
    interrupt:
    
      fatal kernel trap:
         exception       = 0x300 (data storage interrupt)
         virtual address = 0xc003317ca6022a00
         dsisr           = 0x40000000
         srr0            = 0xc000000000f59460 (__syncicache)
         lr              = 0xc000000000f23588 (mmu_radix_sync_icache)
                pid = 23878, comm = skyframe-evaluator-
      panic: data storage interrupt trap
    
    The faulting addresses decode to physical addresses far beyond installed
    memory (~140 TB and ~900 TB on a 256 GB machine), i.e. translations that
    never existed.
    
    The hash MMU implementation of the same method, moea64_sync_icache(),
    already holds PMAP_LOCK() across the loop; do the same here.
    mmu_radix_extract() does not acquire the pmap lock itself, so this
    introduces no recursion.
    
    JIT workloads reach this path constantly: ppc_instr_emulate() calls
    pmap_sync_icache() on the faulting address for the SIGILL "second
    chance" retry, so a multithreaded JVM executing freshly written code
    races against its own threads' mmap/munmap.  Every panic observed here
    was in a JVM thread.
    
    Tested on POWER9 (radix MMU) with a bazel/JVM build loop that previously
    panicked the machine twice within ten minutes: afterwards 13 consecutive
    builds and more than 10 hours of uptime with no panic, on both
    15.1-RELEASE and 16.0-CURRENT.
    
    MFC after:      1 week
    Differential Revision:  https://reviews.freebsd.org/D59311
    Reviewed by:    jhibbits, adrian
---
 sys/powerpc/aim/mmu_radix.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/sys/powerpc/aim/mmu_radix.c b/sys/powerpc/aim/mmu_radix.c
index 7df54408838e..0d958f0ebc5e 100644
--- a/sys/powerpc/aim/mmu_radix.c
+++ b/sys/powerpc/aim/mmu_radix.c
@@ -5978,6 +5978,7 @@ mmu_radix_sync_icache(pmap_t pm, vm_offset_t va, vm_size_t sz)
 	if (__predict_false(pm == NULL))
 		pm = &curthread->td_proc->p_vmspace->vm_pmap;
 
+	PMAP_LOCK(pm);
 	while (sz > 0) {
 		pa = pmap_extract(pm, va);
 		sync_sz = PAGE_SIZE - (va & PAGE_MASK);
@@ -5989,6 +5990,7 @@ mmu_radix_sync_icache(pmap_t pm, vm_offset_t va, vm_size_t sz)
 		va += sync_sz;
 		sz -= sync_sz;
 	}
+	PMAP_UNLOCK(pm);
 }
 
 static __inline void