From nobody Mon Nov 16 09:10:22 2026 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4jShsV2xw6z71ZWc for ; Mon, 16 Nov 2026 09:10:22 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4jShsV2M6Rz4RKQ for ; Mon, 16 Nov 2026 09:10:22 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1794820222; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=ZyDSZipS1m4AqtfrBoSptK5ds3jEvw56PCR4kkN2UG0=; b=uF6epUlmRAoYOyMZx2o8VgjCDQdszXuF9IuB21EXE420bGJIGcf0/geXTO3C/R3Vlk9bBO nVoyp7Yhb9eA/TTaiTKqgU3TbRTN9HkfV0Rxh3penKOnFsTe1Lg0TAtUwxNWQS01beQiIK 4Yd/L8PqbNIHSVhD7+aC8O0MfE4Z+pMjVoKR1VV5NOfSHa6oL2Qd4FahqqGJsvJI40E15x HJJVLUUf8qGEwbl81Ah8hJzuTdw4jiOHes3QBemJIFa1f38lIpRNeKPx8dP7a9jTSrqBXt NTdXXGuwbYouqyDMm788cuitlulJUlVUvXEmbqoiCqpgimrTs8sDM1SxnVoDSw== ARC-Seal: i=1; a=rsa-sha256; d=freebsd.org; s=dkim; cv=none; t=1794820222; b=QkIL3xJelBoTJo07K60UxXSyq9d6s5RImfcOBlhxr89jy+taXNgeUG4jx0AKYCtTa1PPdY HeCxf3GARTygiDRcdDLZh69DGqHLGveYO5xvS1OuEc6B4BjcKw9cHPbUtXLwjeBeFkzzkb mUwQNp0Wx/9KGVKhQUt/sRFcifSmQYlvowcklxl5Xwy6c3xl7mWt43qCrUEj4VoseSDadc SmrkkiyPFB/B4kpnaC75jcFVbqPyQdCnhY0UM0N4F4ZxgbrF830NQ9bLIRiob8PsmotPJS a+mjd7KQ+q8BfTm0BbbES3t9WOKWmhpgSlCAgBQ3vkazVR8uo1dhg8n5E3Kk+g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1794820222; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=ZyDSZipS1m4AqtfrBoSptK5ds3jEvw56PCR4kkN2UG0=; b=OjBTMTtw/POhjhv4wXEzQal/E/1vKuvOeOmVbEN/OwlNi4WYHpd5bLJMrQmE0HBHywvJ/L zT9tds9uSXdky4z49+AumQGMeMLtBidEVpfGIrSa0G4lbfEmqawHgSMUcOGUH6pHqnybTi GtsP01ChUGwD9kHFQz9JEplCpChNVKXAuaVFyIejhUW/8K5bQp9joog0jElq7onLA1ZaL7 ifdVMmttO/sNsq0XBPSKCRAh+/5FXzrb4ALhbhF+7rjhfsz4kmepLV9UU9Z3hbFOcC33yu N5Y704Q6zCHPdTz98tH6UWxRqUvQFfFF6GASHKUL4/c0pzZrFbgr7t3LmksSxQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4jShsV1TJlznNy for ; Mon, 16 Nov 2026 09:10:22 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 6584 by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Mon, 16 Nov 2026 09:10:22 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Piotr Kubaj Subject: git: afeeac4f9cb0 - stable/15 - powerpc/pmap: Use dcbz to zero pages in the radix pmap List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all X-BeenThere: dev-commits-src-all@freebsd.org List-Id: Commit messages for all branches of the src repository List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: pkubaj X-Git-Repository: src X-Git-Refname: refs/heads/stable/15 X-Git-Reftype: branch X-Git-Commit: afeeac4f9cb08a2ea15dfd94f90486a6ddf29d88 Auto-Submitted: auto-generated Date: Mon, 16 Nov 2026 09:10:22 +0000 Message-Id: <6afac87e.6584.52b8f99d@gitrepo.freebsd.org> The branch stable/15 has been updated by pkubaj: URL: https://cgit.FreeBSD.org/src/commit/?id=afeeac4f9cb08a2ea15dfd94f90486a6ddf29d88 commit afeeac4f9cb08a2ea15dfd94f90486a6ddf29d88 Author: Piotr Kubaj AuthorDate: 2026-09-08 11:22:58 +0000 Commit: Piotr Kubaj CommitDate: 2026-11-16 09:10:19 +0000 powerpc/pmap: Use dcbz to zero pages in the radix pmap pagezero() was a plain store loop (bzero), which under the kernel build flags (-mno-vsx -msoft-float) compiles to byte stores. dcbz establishes a zeroed cache line directly in the cache without a read-for-ownership fetch from memory, roughly halving the memory transactions of page zeroing. Measured on POWER9 (Raptor Blackbird, DD2.3, bare metal), zeroing a cold 256 MB buffer with 128-byte scalar loops: byte stores (current libkern memset) 8.6 GB/s doubleword (std) stores 26.8 GB/s dcbz 34.6 GB/s dcbz raises an alignment interrupt on caching-inhibited mappings, and the kernel does not emulate it, so mmu_radix_zero_page() falls back to bzero() for any page whose memattr is not the write-back default. The internal pagezero() callers only touch freshly allocated page-table pages, which are always write-back. Note: dcbz helps only zeroing, where there is no source to read. For page copying it is a pessimization (it adds a redundant zeroing pass on top of the mandatory source read), so mmu_radix_copy_page() is left as a plain bcopy(). Differential Revision: https://reviews.freebsd.org/D59507 Reviewed by: jhibbits (cherry picked from commit 1d4a00d455617e150da5a7c3076579a7e612d33d) --- sys/powerpc/aim/mmu_radix.c | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/sys/powerpc/aim/mmu_radix.c b/sys/powerpc/aim/mmu_radix.c index f1f1a06ecd67..83af83b14bb2 100644 --- a/sys/powerpc/aim/mmu_radix.c +++ b/sys/powerpc/aim/mmu_radix.c @@ -891,12 +891,22 @@ pmap_resident_count_dec(pmap_t pmap, int count) pmap->pm_stats.resident_count -= count; } +/* + * dcbz establishes a zeroed cache line without fetching it from memory, + * but raises an alignment interrupt on caching-inhibited mappings, so it + * may only be used on write-back memory. The internal callers only zero + * freshly allocated page table pages, which are always write-back; the + * vm_page facing entry point guards on the page's memattr. + */ static void pagezero(vm_offset_t va) { - va = trunc_page(va); + vm_offset_t off; - bzero((void *)va, PAGE_SIZE); + va = trunc_page(va); + for (off = 0; off < PAGE_SIZE; off += cacheline_size) + __asm __volatile("dcbz 0,%0" :: + "r"(va + off) : "memory"); } static uint64_t @@ -5808,7 +5818,10 @@ mmu_radix_zero_page(vm_page_t m) CTR2(KTR_PMAP, "%s(%p)", __func__, m); addr = PHYS_TO_DMAP(VM_PAGE_TO_PHYS(m)); - pagezero(addr); + if (__predict_true(m->md.mdpg_cache_attrs == VM_MEMATTR_DEFAULT)) + pagezero(addr); + else + bzero((void *)addr, PAGE_SIZE); } void