From nobody Thu Sep 17 08:08:22 2026 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4hlpKf455xz6sfQ3 for ; Thu, 17 Sep 2026 08:08:22 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "YR2" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4hlpKf1rwHz51K2 for ; Thu, 17 Sep 2026 08:08:22 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1789632502; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=j96ptBD9MMAyVv00IVWsYK5Wpd9I4a3MNkbbknIi5oQ=; b=U6QxlE2dZw8YqMbQKEnG0nf3dtT8aU+oMUiGQspFffS+Zy7xaL+gGDxc81AcxHzq1i2PIj dWAIK/RNyLa19bpHP95h7bFmD4ExnG7yoBM7jlnC+FG+7e8RMu0UECWE82+NTFmIYIlSYR VadhieUk7ZvKRbjqkIHWR4UBGQ0itfDkkT0ngd36s41KNy05PGXOOpzttch9L4EoPusb93 rZLHM4nPBz9K9NwgJXkxcYSULQHJa+w9t/YxnL192xipkQgMtbg2Dl3TerRPhAaN1v+u2C V4JqzX19BOarfl0aF69LL+SvV+S66QkgkSnY5+5yB19WtCL5mGB3jvlcCJKhfQ== ARC-Seal: i=1; a=rsa-sha256; d=freebsd.org; s=dkim; cv=none; t=1789632502; b=mMMPHtwyZBAMA5Hgj4J945PLyFLstGgDqYd804ZtCoFOZKrH0JBcYzhfF8YnXuZoo2UWwW 87FISOG6gl6VVhvnnU8uyEadNv7aixjaKBj/C/Fz2d1zf6I/sv6yEylzOTooiyrJtoKqF/ ko7YzIYzk0EWitTPfY7Y3MGAgqQRhx5t0aLSjJ+rDpl9iXpAsI1hn1Olt3D9e2pS06XwwD gyqnobGMoF0Mzoczzt5zydSoCbBiShWawW98GCi+kUsmIwhqeqOz4tD0PAqeHoX0B16mGv 0OOF0KWEtK5mvrsJ4E4sXjq1jJ7VH0i0LHFV3PYlM9WBs1NJDFZZkloEqOEUDQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1789632502; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=j96ptBD9MMAyVv00IVWsYK5Wpd9I4a3MNkbbknIi5oQ=; b=pqptLELy2E9R3Ti6MxHQjelDq8e41UMEx+CQHFk6gGOrrYmPxlPXqeziFn2Tulr2CI4IHi VFro6yG9+YI0DGyVqk++rPDkohsF2sfjZStnisyC6DyMZK4SQzIcoDurF96C/4L8ianwrc XTfeKwZIk7QDoehwBt8PQEr+RH3EZFBhyPjNQNyskF1q9HytM1aAZV8BAPTgzwSVp1Mzju k9NSShGLOjzbbzNhUXfhUsTKm3p3yp5SYXnEkCiPGBzxGwIb6ds5i9x8o9SG6zmW560RLz FdlYFJCRlYSgB1wTx+L1Z4ZIxJhkqNGxUY+DrE2grIa/21Y1ZtSPPXvfSI51VA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4hlpKf0wFNzNCG for ; Thu, 17 Sep 2026 08:08:22 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 38bf0 by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Thu, 17 Sep 2026 08:08:22 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Piotr Kubaj Subject: git: 1d4a00d45561 - main - powerpc/pmap: Use dcbz to zero pages in the radix pmap List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: pkubaj X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 1d4a00d455617e150da5a7c3076579a7e612d33d Auto-Submitted: auto-generated Date: Thu, 17 Sep 2026 08:08:22 +0000 Message-Id: <6aab9ff6.38bf0.10f49c31@gitrepo.freebsd.org> The branch main has been updated by pkubaj: URL: https://cgit.FreeBSD.org/src/commit/?id=1d4a00d455617e150da5a7c3076579a7e612d33d commit 1d4a00d455617e150da5a7c3076579a7e612d33d Author: Piotr Kubaj AuthorDate: 2026-09-08 11:22:58 +0000 Commit: Piotr Kubaj CommitDate: 2026-09-17 08:08:11 +0000 powerpc/pmap: Use dcbz to zero pages in the radix pmap pagezero() was a plain store loop (bzero), which under the kernel build flags (-mno-vsx -msoft-float) compiles to byte stores. dcbz establishes a zeroed cache line directly in the cache without a read-for-ownership fetch from memory, roughly halving the memory transactions of page zeroing. Measured on POWER9 (Raptor Blackbird, DD2.3, bare metal), zeroing a cold 256 MB buffer with 128-byte scalar loops: byte stores (current libkern memset) 8.6 GB/s doubleword (std) stores 26.8 GB/s dcbz 34.6 GB/s dcbz raises an alignment interrupt on caching-inhibited mappings, and the kernel does not emulate it, so mmu_radix_zero_page() falls back to bzero() for any page whose memattr is not the write-back default. The internal pagezero() callers only touch freshly allocated page-table pages, which are always write-back. Note: dcbz helps only zeroing, where there is no source to read. For page copying it is a pessimization (it adds a redundant zeroing pass on top of the mandatory source read), so mmu_radix_copy_page() is left as a plain bcopy(). Differential Revision: https://reviews.freebsd.org/D59507 Reviewed by: jhibbits --- sys/powerpc/aim/mmu_radix.c | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/sys/powerpc/aim/mmu_radix.c b/sys/powerpc/aim/mmu_radix.c index 7d456a587d29..145b0ec5560b 100644 --- a/sys/powerpc/aim/mmu_radix.c +++ b/sys/powerpc/aim/mmu_radix.c @@ -882,12 +882,22 @@ pmap_resident_count_dec(pmap_t pmap, int count) pmap->pm_stats.resident_count -= count; } +/* + * dcbz establishes a zeroed cache line without fetching it from memory, + * but raises an alignment interrupt on caching-inhibited mappings, so it + * may only be used on write-back memory. The internal callers only zero + * freshly allocated page table pages, which are always write-back; the + * vm_page facing entry point guards on the page's memattr. + */ static void pagezero(void *va) { - va = trunc_page(va); + vm_offset_t off; - bzero(va, PAGE_SIZE); + va = trunc_page(va); + for (off = 0; off < PAGE_SIZE; off += cacheline_size) + __asm __volatile("dcbz 0,%0" :: + "r"((char *)va + off) : "memory"); } static uint64_t @@ -5791,7 +5801,10 @@ mmu_radix_zero_page(vm_page_t m) CTR2(KTR_PMAP, "%s(%p)", __func__, m); addr = VM_PAGE_TO_DMAP(m); - pagezero(addr); + if (__predict_true(m->md.mdpg_cache_attrs == VM_MEMATTR_DEFAULT)) + pagezero(addr); + else + bzero(addr, PAGE_SIZE); } void