From nobody Sat Nov 16 19:17:03 2024 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XrNtR6rrYz5djN1; Sat, 16 Nov 2024 19:17:03 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XrNtR6HPzz4bCW; Sat, 16 Nov 2024 19:17:03 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1731784623; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=lBCHw95qwkkwBZhzICmXyVVxYRi/FxE/HTti2yw9pZg=; b=Db9+CczPrLlSI0g0u0PuAIhCXyj8KTsw9aFh9zMAlBEaFnhvV7fJeGcnIqHnq8taAlx1zk XUPlsONege2IQiJnZfnCN6aZrkgbJIQmZOjDZZOsTLuf1O+O4fqmyeceMBgvfoi29ZJKdi 7Cik798mkt847k0el0P8tXlXqy+u2K7OnveMsmdAclwaxrF8s9iF966uQHz6DYl/Ttoyep 6a9NInxOCtu4N9x0WGogxDUg9EIGyf1gj2OVUpqDNdNwar4xyKkQsEKt//6rnTwAE9a5XW PGMWU9mNi4JDwyEipHTVwAdg74Ut3DR9UWyUB3A/U6bA+f0+jCSUBhT/PlhHeg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1731784623; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=lBCHw95qwkkwBZhzICmXyVVxYRi/FxE/HTti2yw9pZg=; b=fpLIgVOYgqMKo5uQkGkqny8rLkp21Yp1wLydmTHW3CdVm2MaJcixnYccP4t9/iLoNw9/Z9 UQuJjxR/ZXEh7Uuf7bt8JI9O65/p/5nhVwFB8aMxCvNo4/KVsGqC0LtezygC+UVPFq8F7p ozHolzbglmOgc+i3sptMRPErVQTLjveU8sjFyVWrhddePPYLzT0IzdqkmA5OljbZYW4DjX oZx6VSUpb+sAKk5C3FmyARWd21bDB6RXkgp5IPR13EDOZIMpQp85Padww90+d+ljANwAXM rh+w4C69+mh5u/+Bdh3h1UqasyMaJVfPm+a1fKACzJ2WXzxcpFmBrzHqYMt++w== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1731784623; a=rsa-sha256; cv=none; b=Xw7fxz4I7PiZ3um+oYYZM8AUZycm1z6hFsNOdT+jBWQQLLJqWC6tCAo7MLCJdaDgjEqsJK T2cafYPiZlm0eH5Wagu/NcHihkTjk1XYXsQSmhPa0uECEaK/3juuMC+yFZZ+Mtd5spLi+c 2Ix4KFpckl+fvbjMjJpyZxXjAMByhKnZNWROOGjguBHxMHr4/bOKbkIEmKLh5R+/9potGY BMcQwyjL5BpyVZylbL9T48GBUaCUbF/OpUyxYQC7+vbxlyNKjZa60OMxgdIPwgbSWyR4MF qMVMEidNcRdKzhkvJAHKad3vDOxAwd2V5FdXpPWTqa66wGlvjOP+C5pNXzaZjw== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4XrNtR5tDPzKW1; Sat, 16 Nov 2024 19:17:03 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 4AGJH3jU040426; Sat, 16 Nov 2024 19:17:03 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 4AGJH38Y040423; Sat, 16 Nov 2024 19:17:03 GMT (envelope-from git) Date: Sat, 16 Nov 2024 19:17:03 GMT Message-Id: <202411161917.4AGJH38Y040423@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Doug Moore Subject: git: f334c0b8b3cb - main - vm_page: use iterators in alloc_contig_domain List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: dougm X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: f334c0b8b3cb97ada3d692a1d208b6839cd1d059 Auto-Submitted: auto-generated The branch main has been updated by dougm: URL: https://cgit.FreeBSD.org/src/commit/?id=f334c0b8b3cb97ada3d692a1d208b6839cd1d059 commit f334c0b8b3cb97ada3d692a1d208b6839cd1d059 Author: Doug Moore AuthorDate: 2024-11-16 19:15:05 +0000 Commit: Doug Moore CommitDate: 2024-11-16 19:15:05 +0000 vm_page: use iterators in alloc_contig_domain Restructure a bit of code to allow vm_page_alloc_contig_domain to use pctrie iterators for lookup and insertion into the object radix tree, to improve performance. Reviewed by: alc Differential Revision: https://reviews.freebsd.org/D47036 --- sys/vm/vm_page.c | 46 +++++++++++++++++++++++++++++++++++++--------- 1 file changed, 37 insertions(+), 9 deletions(-) diff --git a/sys/vm/vm_page.c b/sys/vm/vm_page.c index 57e5684b3178..6b49f0745c73 100644 --- a/sys/vm/vm_page.c +++ b/sys/vm/vm_page.c @@ -1475,16 +1475,17 @@ vm_page_dirty_KBI(vm_page_t m) /* * Insert the given page into the given object at the given pindex. mpred is * used for memq linkage. From vm_page_insert, lookup is true, mpred is - * initially NULL, and this procedure looks it up. From vm_page_insert_after, - * lookup is false and mpred is known to the caller to be valid, and may be - * NULL if this will be the page with the lowest pindex. + * initially NULL, and this procedure looks it up. From vm_page_insert_after + * and vm_page_iter_insert, lookup is false and mpred is known to the caller + * to be valid, and may be NULL if this will be the page with the lowest + * pindex. * * The procedure is marked __always_inline to suggest to the compiler to * eliminate the lookup parameter and the associated alternate branch. */ static __always_inline int vm_page_insert_lookup(vm_page_t m, vm_object_t object, vm_pindex_t pindex, - vm_page_t mpred, bool lookup) + struct pctrie_iter *pages, bool iter, vm_page_t mpred, bool lookup) { int error; @@ -1503,7 +1504,10 @@ vm_page_insert_lookup(vm_page_t m, vm_object_t object, vm_pindex_t pindex, * Add this page to the object's radix tree, and look up mpred if * needed. */ - if (lookup) + if (iter) { + KASSERT(!lookup, ("%s: cannot lookup mpred", __func__)); + error = vm_radix_iter_insert(pages, m); + } else if (lookup) error = vm_radix_insert_lookup_lt(&object->rtree, m, &mpred); else error = vm_radix_insert(&object->rtree, m); @@ -1532,7 +1536,8 @@ vm_page_insert_lookup(vm_page_t m, vm_object_t object, vm_pindex_t pindex, int vm_page_insert(vm_page_t m, vm_object_t object, vm_pindex_t pindex) { - return (vm_page_insert_lookup(m, object, pindex, NULL, true)); + return (vm_page_insert_lookup(m, object, pindex, NULL, false, NULL, + true)); } /* @@ -1549,7 +1554,28 @@ static int vm_page_insert_after(vm_page_t m, vm_object_t object, vm_pindex_t pindex, vm_page_t mpred) { - return (vm_page_insert_lookup(m, object, pindex, mpred, false)); + return (vm_page_insert_lookup(m, object, pindex, NULL, false, mpred, + false)); +} + +/* + * vm_page_iter_insert: + * + * Tries to insert the page "m" into the specified object at offset + * "pindex" using the iterator "pages". Returns 0 if the insertion was + * successful. + * + * The page "mpred" must immediately precede the offset "pindex" within + * the specified object. + * + * The object must be locked. + */ +static int +vm_page_iter_insert(struct pctrie_iter *pages, vm_page_t m, vm_object_t object, + vm_pindex_t pindex, vm_page_t mpred) +{ + return (vm_page_insert_lookup(m, object, pindex, pages, true, mpred, + false)); } /* @@ -2373,6 +2399,7 @@ vm_page_alloc_contig_domain(vm_object_t object, vm_pindex_t pindex, int domain, int req, u_long npages, vm_paddr_t low, vm_paddr_t high, u_long alignment, vm_paddr_t boundary, vm_memattr_t memattr) { + struct pctrie_iter pages; vm_page_t m, m_ret, mpred; u_int busy_lock, flags, oflags; @@ -2391,7 +2418,8 @@ vm_page_alloc_contig_domain(vm_object_t object, vm_pindex_t pindex, int domain, object)); KASSERT(npages > 0, ("vm_page_alloc_contig: npages is zero")); - mpred = vm_page_mpred(object, pindex); + vm_page_iter_init(&pages, object); + mpred = vm_radix_iter_lookup_le(&pages, pindex); KASSERT(mpred == NULL || mpred->pindex != pindex, ("vm_page_alloc_contig: pindex already allocated")); for (;;) { @@ -2440,7 +2468,7 @@ vm_page_alloc_contig_domain(vm_object_t object, vm_pindex_t pindex, int domain, m->ref_count = 1; m->a.act_count = 0; m->oflags = oflags; - if (vm_page_insert_after(m, object, pindex, mpred)) { + if (vm_page_iter_insert(&pages, m, object, pindex, mpred)) { if ((req & VM_ALLOC_WIRED) != 0) vm_wire_sub(npages); KASSERT(m->object == NULL,