From nobody Thu Nov 19 06:42:09 2026 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4jVTR60cQgz71tLB for ; Thu, 19 Nov 2026 06:42:10 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4jVTR54yf1z4Jsg for ; Thu, 19 Nov 2026 06:42:09 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1795070529; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=WeZCK1d15KTofa41f8bUW7EhzrkgKjWjgz0gIMxXPXw=; b=RRh8DOTjyF4u2/JXtx8uLApP5MsoinKBA0d3pM9SlyC00LthRvdFFCOYqoA6U6Wx0BtOx2 kSIaGeYlT+++gQOOs3Vomy4tgbQ4YSv8yBTaBHQA6KEKCm3L4K7HT0hbX8LyRmlWt4kd/1 76DPCeulRT5n2MOtSOGizqCkbpaSYmDY4cArgKcM2lI9XosACyfabLOOT/K2DPs7JCIKU9 ydPp16zKVCJWRV9Ounnw4PVInnfwgpfNFSTtBtzQKzsBYCEwO96El6IekbzlAgjjOIkHNP vm7knkAeBJeySoGe5hvQNXOKVipG+R+c8fwS+hLA82jGxfBjHZzklNxCnd0V1A== ARC-Seal: i=1; a=rsa-sha256; d=freebsd.org; s=dkim; cv=none; t=1795070529; b=EdoNdhKbEDpgun3b1r2IrFT/tdIUBMK2CoUquoPP0wRL6sZ4v4Ne8ZH+87VzfdtDA/MdRl 72PQ3X9n4TkH//oj2crp008SLix8OvPUb/KDtyZHtJJgqxl0fduVsBs7fRsl2lv3iMf+wF 0zoHcgJguOc6WsXQ96M5vLZgNu/p9FTy98MzN/HcHSMpDnIXF+qYaCr+7rkZPahdEd0eV3 Lrj/8VBbnoX17+q5VjcURJ9HJC9fXd+3IULy7qlA6CSUGS9Si073R6WkqDNIYcOK7K/M7n 4XzaGcX+Tx5fggRVL3TG54O5zZ4hkFPnynz300Ky+hcrk7YKiLBLU9mKcCdRjw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1795070529; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=WeZCK1d15KTofa41f8bUW7EhzrkgKjWjgz0gIMxXPXw=; b=SJCuMr32WgmNbakV/NY+qkFa99DQyIm0Agnq9UXXSbe/FbbX0pyUPYmA5LaQ5ADm5RnMMw IroqBfBYsAbSorqs8OabPmvxDLVz0YPNm4NM4XWEMzE/upNsu2l4+wEeIvaHa1SIWTY7Jf 26Ci9Fl3Jq8WCdBnQmlZ72Nrd58t9KmkBCsaW79wkTvpvWAKEXA1wQH+UMG2boQ7ONfS5s u0ZJSaLUk65FN3jjUbPxaVrFlMoK+AYWWz9eDWBO4YiReArQ4dE91aqONOOTDNuhoaC8M1 MP7JUpyxAK11dKI1xpNfNZhVan42Xa6S406rGnci7vq8NVoia7T4SkXIFcVXPg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4jVTR54Yffzpx2 for ; Thu, 19 Nov 2026 06:42:09 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 1aef3 by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Thu, 19 Nov 2026 06:42:09 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Alan Cox Subject: git: ad0cd0101590 - main - arm64 pmap: Eliminate page table rewalks after demotions List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all X-BeenThere: dev-commits-src-all@freebsd.org List-Id: Commit messages for all branches of the src repository List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: alc X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: ad0cd01015906ce421cd2c3b7251a0f615bc0a20 Auto-Submitted: auto-generated Date: Thu, 19 Nov 2026 06:42:09 +0000 Message-Id: <6afe9a41.1aef3.31f17fb4@gitrepo.freebsd.org> The branch main has been updated by alc: URL: https://cgit.FreeBSD.org/src/commit/?id=ad0cd01015906ce421cd2c3b7251a0f615bc0a20 commit ad0cd01015906ce421cd2c3b7251a0f615bc0a20 Author: Alan Cox AuthorDate: 2026-11-09 17:20:02 +0000 Commit: Alan Cox CommitDate: 2026-11-19 06:40:23 +0000 arm64 pmap: Eliminate page table rewalks after demotions pmap_demote_l1() and pmap_demote_l2_locked() return the virtual address of the page table page that replaces the demoted superpage mapping. In pmap_advise(), pmap_change_props_locked(), and pmap_clear_modify() use that address instead of rewalking the page table from the L1 or L2 entry. Similarly, after pmap_advise() demotes an L3C superpage, compute the virtual address of the L3 entry that it removes from the virtual address of the current L3 entry. Reviewed by: kib MFC after: 1 week Differential Revision: https://reviews.freebsd.org/D61599 --- sys/arm64/arm64/pmap.c | 30 +++++++++++++++++------------- 1 file changed, 17 insertions(+), 13 deletions(-) diff --git a/sys/arm64/arm64/pmap.c b/sys/arm64/arm64/pmap.c index c4435d7dd3eb..7340627e6721 100644 --- a/sys/arm64/arm64/pmap.c +++ b/sys/arm64/arm64/pmap.c @@ -8242,7 +8242,8 @@ pmap_advise(pmap_t pmap, vm_offset_t sva, vm_offset_t eva, int advice) if ((oldl2 & ATTR_SW_MANAGED) == 0) continue; lock = NULL; - if (!pmap_demote_l2_locked(pmap, l2, sva, &lock)) { + l3 = pmap_demote_l2_locked(pmap, l2, sva, &lock); + if (l3 == NULL) { if (lock != NULL) rw_wunlock(lock); @@ -8268,7 +8269,7 @@ pmap_advise(pmap_t pmap, vm_offset_t sva, vm_offset_t eva, int advice) va -= PAGE_SIZE; KASSERT(va >= sva, ("pmap_advise: no address gap")); - l3 = pmap_l2_to_l3(l2, va); + l3 = &l3[pmap_l3_index(va)]; KASSERT(pmap_load(l3) != 0, ("pmap_advise: invalid PTE")); pmap_remove_l3(pmap, l3, va, pmap_load(l2), @@ -8314,10 +8315,14 @@ pmap_advise(pmap_t pmap, vm_offset_t sva, vm_offset_t eva, int advice) * repromotion trigger. */ if ((oldl3 & ATTR_SW_WIRED) == 0) { + /* + * sva and l3 may not be at the + * start of the L3C superpage. + */ dva = MIN((sva & ~L3C_OFFSET) + L3C_SIZE - PAGE_SIZE, va_next - PAGE_SIZE); - dl3 = pmap_l2_to_l3(l2, dva); + dl3 = &l3[atop(dva - sva)]; KASSERT(pmap_load(dl3) != 0, ("pmap_advise: invalid PTE")); lock = NULL; @@ -8410,14 +8415,14 @@ restart: oldl2 = pmap_load(l2); /* If oldl2 has ATTR_SW_DBM set, then it is also dirty. */ if ((oldl2 & ATTR_SW_DBM) != 0 && - pmap_demote_l2_locked(pmap, l2, va, &lock) && + (l3 = pmap_demote_l2_locked(pmap, l2, va, &lock)) != NULL && (oldl2 & ATTR_SW_WIRED) == 0) { /* * Write protect the mapping to a single page so that * a subsequent write access may repromote. */ va += VM_PAGE_TO_PHYS(m) - PTE_TO_PHYS(oldl2); - l3 = pmap_l2_to_l3(l2, va); + l3 = &l3[pmap_l3_index(va)]; if ((pmap_load(l3) & ATTR_CONTIGUOUS) != 0) (void)pmap_demote_l3c(pmap, l3, va); oldl3 = pmap_load(l3); @@ -8760,7 +8765,7 @@ pmap_change_props_locked(void *addr, vm_size_t size, vm_prot_t prot, vm_offset_t base, offset, tmpva, va; vm_size_t pte_size; vm_paddr_t pa; - pt_entry_t pte, *ptep, *newpte; + pt_entry_t pte, *ptep; pt_entry_t bits, mask, old_mode_bits, old_mode_mask; char *tmpptep; int lvl, rv; @@ -8857,11 +8862,11 @@ pmap_change_props_locked(void *addr, vm_size_t size, vm_prot_t prot, pte_size = L1_SIZE; break; } - newpte = pmap_demote_l1(kernel_pmap, ptep, + ptep = pmap_demote_l1(kernel_pmap, ptep, tmpva & ~L1_OFFSET); - if (newpte == NULL) + if (ptep == NULL) return (EINVAL); - ptep = pmap_l1_to_l2(ptep, tmpva); + ptep = &ptep[pmap_l2_index(tmpva)]; /* FALLTHROUGH */ case 2: if ((pmap_load(ptep) & ATTR_CONTIGUOUS) != 0) { @@ -8879,11 +8884,10 @@ pmap_change_props_locked(void *addr, vm_size_t size, vm_prot_t prot, pte_size = L2_SIZE; break; } - newpte = pmap_demote_l2(kernel_pmap, ptep, - tmpva); - if (newpte == NULL) + ptep = pmap_demote_l2(kernel_pmap, ptep, tmpva); + if (ptep == NULL) return (EINVAL); - ptep = pmap_l2_to_l3(ptep, tmpva); + ptep = &ptep[pmap_l3_index(tmpva)]; /* FALLTHROUGH */ case 3: if ((pmap_load(ptep) & ATTR_CONTIGUOUS) != 0) {