From nobody Thu Apr 24 19:16:20 2025 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Zk5LD4fjVz5tgwZ; Thu, 24 Apr 2025 19:16:20 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Zk5LD31mGz44d7; Thu, 24 Apr 2025 19:16:20 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1745522180; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=dTkrsq+AwirWNG6l12JxyhgMkFEJFax8LSzlGMCpO9I=; b=FzWq/SBPIqCwPjpKnNtxOguSN+CkdMZnOSaFvoo61sHtyzz16sYu1yGCeAVVPiUIpiRDIq MhT5RzQLhBZW4SGVIL8JacuptAybxFRHVwXTMw5I8S0tBvZcjtbrZ7muGBSl4umXajk2Of Na9jny7wCaFiA6MIekiNyJVPmWLurYSqJVqs9xyZrHv23q0La8EDNohWhAIq1D+zd6z4b7 gJUmcpvIsXLMhiOYCxqVR25bLhwpsGYZuh6bhmybvdJ5HNIvfvbXqujzHEoPaYKBriLrdD sWoyJphYAIbJDM94HarUhJ2vyZvbCcfETySokc4sV84q5w5wwnJXUa0L6MqtJg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1745522180; a=rsa-sha256; cv=none; b=FQW2UA62BLB69LfMEvbLFLbE7h2JHMPGqINBR9ycEnMI7nGVf1c2WE0C5ZxRUTQ+Vdmmm+ PDTe0YPbZINN7Sr8ysDvD/6Xa4fuUQzAN4zwtr/shrjTnPTa+4XgIyYDYzchMSX8gk88Aj 9IPufmNpONgY2f65830SBwyZts3FXU690TgRLY8jNfuaCNcQrncP/fy3iu/sqCsoxYYJmn BDfKB7oDuDsqfnSAsRDRYaEt0JgI7RHj97TdkqKRpL9LguI1w/YhE9diKt5r5hI20NAD7z Mq/3bhxQfHQe96lBSktOeX5QJrIjmuKUI8ZCxULOlWQuurEuGNXwyEXLYFh4mg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1745522180; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=dTkrsq+AwirWNG6l12JxyhgMkFEJFax8LSzlGMCpO9I=; b=pSFNZTSxgArRi9cjWGxjHyexVWXSpI6LQkD5TiUhXd1jbhKEGaRpCYUULtOvvw3+Q33H14 s/27s6xJNS6TG9RYV0lIEGUcE6lIagBsxfLf15bxOScQKia7jEkKFY3TEXiuUcn2mhK5X4 +586I5svIl69yT5Xjs7fw2L+LI1Rw5YWxGdqYKCiYxTOrPTshdGwNPEvc2v0CKjbzzZPEU ex2vK7mVAeHvSUUW+VbdSeF/VqzX8T/EcLn3xIzPK5TexIGmnmlLkg4VuibVfDtoewo2dd VXQFtz/MS4vHx9BXEcPu/ef4oSOFNJdudureBE6/wBZF3aHmS703yfuqel7pTA== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4Zk5LD2HjFz11gq; Thu, 24 Apr 2025 19:16:20 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 53OJGKZc000683; Thu, 24 Apr 2025 19:16:20 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 53OJGKJO000680; Thu, 24 Apr 2025 19:16:20 GMT (envelope-from git) Date: Thu, 24 Apr 2025 19:16:20 GMT Message-Id: <202504241916.53OJGKJO000680@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Doug Moore Subject: git: 896dccb1c122 - main - vm_page: make grab_valid() with iter param List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: dougm X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 896dccb1c1225d13ecb88a872204e2f80bcda453 Auto-Submitted: auto-generated The branch main has been updated by dougm: URL: https://cgit.FreeBSD.org/src/commit/?id=896dccb1c1225d13ecb88a872204e2f80bcda453 commit 896dccb1c1225d13ecb88a872204e2f80bcda453 Author: Doug Moore AuthorDate: 2025-04-24 19:14:45 +0000 Commit: Doug Moore CommitDate: 2025-04-24 19:14:45 +0000 vm_page: make grab_valid() with iter param Define vm_page_grab_valid_iter() to be the function that does the work of vm_page_grab(), but has an iterator parameter passed to it. In a couple of places, this can be called directly to let an iterator get reused. Reviewed by: markj Differential Revision: https://reviews.freebsd.org/D49942 --- sys/kern/uipc_shm.c | 7 +++++-- sys/vm/vm_page.c | 51 ++++++++++++++++++++++++++++++++++++--------------- sys/vm/vm_page.h | 2 ++ 3 files changed, 43 insertions(+), 17 deletions(-) diff --git a/sys/kern/uipc_shm.c b/sys/kern/uipc_shm.c index 7004e4f1c6ea..28e90b73aa8d 100644 --- a/sys/kern/uipc_shm.c +++ b/sys/kern/uipc_shm.c @@ -100,6 +100,7 @@ #include #include #include +#include #include struct shm_mapping { @@ -195,6 +196,7 @@ SYSCTL_INT(_vm_largepages, OID_AUTO, reclaim_tries, static int uiomove_object_page(vm_object_t obj, size_t len, struct uio *uio) { + struct pctrie_iter pages; vm_page_t m; vm_pindex_t idx; size_t tlen; @@ -214,8 +216,9 @@ uiomove_object_page(vm_object_t obj, size_t len, struct uio *uio) * page: use zero_region. This is intended to avoid instantiating * pages on read from a sparse region. */ + vm_page_iter_init(&pages, obj); VM_OBJECT_WLOCK(obj); - m = vm_page_lookup(obj, idx); + m = vm_radix_iter_lookup(&pages, idx); if (uio->uio_rw == UIO_READ && m == NULL && !vm_pager_has_page(obj, idx, NULL, NULL)) { VM_OBJECT_WUNLOCK(obj); @@ -229,7 +232,7 @@ uiomove_object_page(vm_object_t obj, size_t len, struct uio *uio) * lock to page out tobj's pages because tobj is a OBJT_SWAP * type object. */ - rv = vm_page_grab_valid(&m, obj, idx, + rv = vm_page_grab_valid_iter(&m, obj, &pages, idx, VM_ALLOC_NORMAL | VM_ALLOC_SBUSY | VM_ALLOC_IGN_SBUSY); if (rv != VM_PAGER_OK) { VM_OBJECT_WUNLOCK(obj); diff --git a/sys/vm/vm_page.c b/sys/vm/vm_page.c index 6601ea23bf0e..9e41735460db 100644 --- a/sys/vm/vm_page.c +++ b/sys/vm/vm_page.c @@ -4946,17 +4946,17 @@ vm_page_grab_unlocked(vm_object_t object, vm_pindex_t pindex, int allocflags) } /* - * Grab a page and make it valid, paging in if necessary. Pages missing from - * their pager are zero filled and validated. If a VM_ALLOC_COUNT is supplied - * and the page is not valid as many as VM_INITIAL_PAGEIN pages can be brought - * in simultaneously. Additional pages will be left on a paging queue but - * will neither be wired nor busy regardless of allocflags. + * Grab a page and make it valid, paging in if necessary. Use an iterator + * parameter. Pages missing from their pager are zero filled and validated. If + * a VM_ALLOC_COUNT is supplied and the page is not valid as many as + * VM_INITIAL_PAGEIN pages can be brought in simultaneously. Additional pages + * will be left on a paging queue but will neither be wired nor busy regardless + * of allocflags. */ int -vm_page_grab_valid(vm_page_t *mp, vm_object_t object, vm_pindex_t pindex, - int allocflags) +vm_page_grab_valid_iter(vm_page_t *mp, vm_object_t object, + struct pctrie_iter *pages, vm_pindex_t pindex, int allocflags) { - struct pctrie_iter pages; vm_page_t m, mpred; vm_page_t ma[VM_INITIAL_PAGEIN]; int after, i, pflags, rv; @@ -4971,10 +4971,9 @@ vm_page_grab_valid(vm_page_t *mp, vm_object_t object, vm_pindex_t pindex, pflags = allocflags & ~(VM_ALLOC_NOBUSY | VM_ALLOC_SBUSY | VM_ALLOC_WIRED | VM_ALLOC_IGN_SBUSY); pflags |= VM_ALLOC_WAITFAIL; - vm_page_iter_init(&pages, object); retrylookup: - if ((m = vm_radix_iter_lookup(&pages, pindex)) != NULL) { + if ((m = vm_radix_iter_lookup(pages, pindex)) != NULL) { /* * If the page is fully valid it can only become invalid * with the object lock held. If it is not valid it can @@ -4988,7 +4987,7 @@ retrylookup: vm_page_all_valid(m) ? allocflags : 0)) { (void)vm_page_grab_sleep(object, m, pindex, "pgrbwt", allocflags, true); - pctrie_iter_reset(&pages); + pctrie_iter_reset(pages); goto retrylookup; } if (vm_page_all_valid(m)) @@ -5002,13 +5001,14 @@ retrylookup: *mp = NULL; return (VM_PAGER_FAIL); } else { - mpred = vm_radix_iter_lookup_lt(&pages, pindex); - m = vm_page_alloc_after(object, &pages, pindex, pflags, mpred); + mpred = vm_radix_iter_lookup_lt(pages, pindex); + m = vm_page_alloc_after(object, pages, pindex, pflags, mpred); if (m == NULL) { if (!vm_pager_can_alloc_page(object, pindex)) { *mp = NULL; return (VM_PAGER_AGAIN); } + pctrie_iter_reset(pages); goto retrylookup; } } @@ -5019,10 +5019,11 @@ retrylookup: after = MIN(after, allocflags >> VM_ALLOC_COUNT_SHIFT); after = MAX(after, 1); ma[0] = mpred = m; + pctrie_iter_reset(pages); for (i = 1; i < after; i++) { - m = vm_radix_iter_lookup(&pages, pindex + i); + m = vm_radix_iter_lookup(pages, pindex + i); if (m == NULL) { - m = vm_page_alloc_after(object, &pages, + m = vm_page_alloc_after(object, pages, pindex + i, VM_ALLOC_NORMAL, mpred); if (m == NULL) break; @@ -5054,6 +5055,7 @@ retrylookup: } else { vm_page_zero_invalid(m, TRUE); } + pctrie_iter_reset(pages); out: if ((allocflags & VM_ALLOC_WIRED) != 0) vm_page_wire(m); @@ -5065,6 +5067,25 @@ out: return (VM_PAGER_OK); } +/* + * Grab a page and make it valid, paging in if necessary. Pages missing from + * their pager are zero filled and validated. If a VM_ALLOC_COUNT is supplied + * and the page is not valid as many as VM_INITIAL_PAGEIN pages can be brought + * in simultaneously. Additional pages will be left on a paging queue but + * will neither be wired nor busy regardless of allocflags. + */ +int +vm_page_grab_valid(vm_page_t *mp, vm_object_t object, vm_pindex_t pindex, + int allocflags) +{ + struct pctrie_iter pages; + + VM_OBJECT_ASSERT_WLOCKED(object); + vm_page_iter_init(&pages, object); + return (vm_page_grab_valid_iter(mp, object, &pages, pindex, + allocflags)); +} + /* * Grab a page. Keep on waiting, as long as the page exists in the object. If * the page doesn't exist, and the pager has it, allocate it and zero part of diff --git a/sys/vm/vm_page.h b/sys/vm/vm_page.h index bf2e850612d7..c24ae8b64d28 100644 --- a/sys/vm/vm_page.h +++ b/sys/vm/vm_page.h @@ -641,6 +641,8 @@ int vm_page_grab_pages_unlocked(vm_object_t object, vm_pindex_t pindex, int allocflags, vm_page_t *ma, int count); int vm_page_grab_valid(vm_page_t *mp, vm_object_t object, vm_pindex_t pindex, int allocflags); +int vm_page_grab_valid_iter(vm_page_t *mp, vm_object_t object, + struct pctrie_iter *, vm_pindex_t pindex, int allocflags); int vm_page_grab_valid_unlocked(vm_page_t *mp, vm_object_t object, vm_pindex_t pindex, int allocflags); void vm_page_deactivate(vm_page_t);