From nobody Thu Apr 17 22:07:24 2025 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4ZdsSr2r0Xz5srt5; Thu, 17 Apr 2025 22:07:24 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4ZdsSr1zSWz3dF3; Thu, 17 Apr 2025 22:07:24 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1744927644; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Gb4XzACowBpl64ubeo2C2ekYp6mzc1CH+vltgcfBjIE=; b=Tq4ivEXFpMjeOqyDUF5nEeB9CYiaVk+LtSqaCfkxbfpdWWYxoqgk7cLfdBjTOBF8Lqo+Kd A1sFG+E4pgtcyxMtbs9V+CMFi7mdKJS+EIu+FYnNQVPes/GOdN02MVqzom7WqEQPoD2YQW RZ9Ythd9fNGRLfSgPEkw/A+Q7DE+1egHnJ9duVwHjMzrLqNBfVorpfqCoD9v5A9LHQrxh6 +OlyvFQtJaIxxoyhArkVNDm2Aj0aZdGNeAL6znn1owo+AyHH7rpxtt1ZmenmTn392Zs5/R PJ/ZxEgY7jg1FEju/B3FbwtUUyEp6W+RiVjmBq6BZ689waTBc9VPWDyFmb1LsA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1744927644; a=rsa-sha256; cv=none; b=BJTkzseont9p+685HrR0oDBNnHzlXxMtKZuuui+xYngpYOU2/ZfMvC2Fn/QTp+K0S967lI 1FlMIPAY+pV9cw1eYq8uiYslwhHurqFVmd+orzGJUe2zNvVbGjO987sJDboP/L0w1M4uGM XZ37W7tfLnnWJWBpUwDR4vH/8myxcfdyA4/lKf22XRzXPro6Zt8ALXsCWvI5ZIufdfxKEj aa233b6bjTtrNDII8SbrjylJ75l5nVc6O3Elo6mpJZ+aq8fHdZSck5UO9hd0j+GkZ+8bq1 +poZhoo67g5tDJAoIOgzZvCIKph4tKpNvGqcCSJvNxLV09LqT3DuWtWgn25upA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1744927644; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Gb4XzACowBpl64ubeo2C2ekYp6mzc1CH+vltgcfBjIE=; b=bfKP5Kf/R/uIIZ6v2Bx4ML2zLPpE/7FI6Bxc2ZoqDYFUfi11QWxqXlETeYLFfFeDRoihOq 94/HiazaiHzlR1WADJr+voJI36M9QzBNqRI278knq1SRY7aM1VukaSVDVjLsQHdfiWJYqI wHUgLWNqKdq7Y7VQsf6zuLrlDpK4SV+1QBta4ojY4FGWCQdXRVZozR1dCe2ZfDCTnpF/Fa YXy4ZTlq03rsgYk7dmd3oR21D8mAwGjKZBc0mu4AotS5C7xdnPF3MoFKxGfPR77w66oZcl NMm5kVfYghRjgX2dpzgUslCb+Xjc+1fD8ZTI6UVReZiRuacHHH1GqXoGnZi4pA== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4ZdsSr1QcHz15n8; Thu, 17 Apr 2025 22:07:24 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 53HM7O0A022331; Thu, 17 Apr 2025 22:07:24 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 53HM7ODV022328; Thu, 17 Apr 2025 22:07:24 GMT (envelope-from git) Date: Thu, 17 Apr 2025 22:07:24 GMT Message-Id: <202504172207.53HM7ODV022328@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Doug Moore Subject: git: 514c2930292e - main - vm_object: use iterators in page_collect_flush() List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: dougm X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 514c2930292ec4e275ea0417ede98b8c78ea8930 Auto-Submitted: auto-generated The branch main has been updated by dougm: URL: https://cgit.FreeBSD.org/src/commit/?id=514c2930292ec4e275ea0417ede98b8c78ea8930 commit 514c2930292ec4e275ea0417ede98b8c78ea8930 Author: Doug Moore AuthorDate: 2025-04-17 22:05:30 +0000 Commit: Doug Moore CommitDate: 2025-04-17 22:05:30 +0000 vm_object: use iterators in page_collect_flush() Implement vm_object_page_collect_flush() using iterators instead of vm_page_next() and vm_page_prev(). Reviewed by: kib Differential Revision: https://reviews.freebsd.org/D49874 --- sys/vm/vm_object.c | 59 ++++++++++++++++++++++++++++++------------------------ 1 file changed, 33 insertions(+), 26 deletions(-) diff --git a/sys/vm/vm_object.c b/sys/vm/vm_object.c index 19e585745a80..c696408def20 100644 --- a/sys/vm/vm_object.c +++ b/sys/vm/vm_object.c @@ -110,8 +110,8 @@ static int old_msync; SYSCTL_INT(_vm, OID_AUTO, old_msync, CTLFLAG_RW, &old_msync, 0, "Use old (insecure) msync behavior"); -static int vm_object_page_collect_flush(vm_object_t object, vm_page_t p, - int pagerflags, int flags, boolean_t *allclean, +static int vm_object_page_collect_flush(struct pctrie_iter *pages, + vm_page_t p, int pagerflags, int flags, boolean_t *allclean, boolean_t *eio); static boolean_t vm_object_page_remove_write(vm_page_t p, int flags, boolean_t *allclean); @@ -1032,6 +1032,7 @@ boolean_t vm_object_page_clean(vm_object_t object, vm_ooffset_t start, vm_ooffset_t end, int flags) { + struct pctrie_iter pages; vm_page_t np, p; vm_pindex_t pi, tend, tstart; int curgeneration, n, pagerflags; @@ -1050,31 +1051,37 @@ vm_object_page_clean(vm_object_t object, vm_ooffset_t start, vm_ooffset_t end, tend = (end == 0) ? object->size : OFF_TO_IDX(end + PAGE_MASK); allclean = tstart == 0 && tend >= object->size; res = TRUE; + vm_page_iter_init(&pages, object); rescan: curgeneration = object->generation; - for (p = vm_page_find_least(object, tstart); p != NULL; p = np) { + for (p = vm_radix_iter_lookup_ge(&pages, tstart); p != NULL; p = np) { pi = p->pindex; if (pi >= tend) break; - np = TAILQ_NEXT(p, listq); - if (vm_page_none_valid(p)) + if (vm_page_none_valid(p)) { + np = vm_radix_iter_step(&pages); continue; - if (vm_page_busy_acquire(p, VM_ALLOC_WAITFAIL) == 0) { + } + if (!vm_page_busy_acquire(p, VM_ALLOC_WAITFAIL)) { if (object->generation != curgeneration && - (flags & OBJPC_SYNC) != 0) + (flags & OBJPC_SYNC) != 0) { + pctrie_iter_reset(&pages); goto rescan; - np = vm_page_find_least(object, pi); + } + np = vm_radix_iter_lookup_ge(&pages, pi); continue; } if (!vm_object_page_remove_write(p, flags, &allclean)) { + np = vm_radix_iter_step(&pages); vm_page_xunbusy(p); continue; } if (object->type == OBJT_VNODE) { - n = vm_object_page_collect_flush(object, p, pagerflags, + n = vm_object_page_collect_flush(&pages, p, pagerflags, flags, &allclean, &eio); + pctrie_iter_reset(&pages); if (eio) { res = FALSE; allclean = FALSE; @@ -1103,7 +1110,7 @@ rescan: n = 1; vm_page_xunbusy(p); } - np = vm_page_find_least(object, pi + n); + np = vm_radix_iter_lookup_ge(&pages, pi + n); } #if 0 VOP_FSYNC(vp, (pagerflags & VM_PAGER_PUT_SYNC) ? MNT_WAIT : 0); @@ -1120,37 +1127,37 @@ rescan: } static int -vm_object_page_collect_flush(vm_object_t object, vm_page_t p, int pagerflags, - int flags, boolean_t *allclean, boolean_t *eio) +vm_object_page_collect_flush(struct pctrie_iter *pages, vm_page_t p, + int pagerflags, int flags, boolean_t *allclean, boolean_t *eio) { - vm_page_t ma[2 * vm_pageout_page_count - 1], tp; + vm_page_t ma[2 * vm_pageout_page_count - 1]; int base, count, runlen; vm_page_lock_assert(p, MA_NOTOWNED); vm_page_assert_xbusied(p); - VM_OBJECT_ASSERT_WLOCKED(object); base = nitems(ma) / 2; ma[base] = p; - for (count = 1, tp = p; count < vm_pageout_page_count; count++) { - tp = vm_page_next(tp); - if (tp == NULL || vm_page_tryxbusy(tp) == 0) + for (count = 1; count < vm_pageout_page_count; count++) { + p = vm_radix_iter_next(pages); + if (p == NULL || vm_page_tryxbusy(p) == 0) break; - if (!vm_object_page_remove_write(tp, flags, allclean)) { - vm_page_xunbusy(tp); + if (!vm_object_page_remove_write(p, flags, allclean)) { + vm_page_xunbusy(p); break; } - ma[base + count] = tp; + ma[base + count] = p; } - for (tp = p; count < vm_pageout_page_count; count++) { - tp = vm_page_prev(tp); - if (tp == NULL || vm_page_tryxbusy(tp) == 0) + pages->index = ma[base]->pindex; + for (; count < vm_pageout_page_count; count++) { + p = vm_radix_iter_prev(pages); + if (p == NULL || vm_page_tryxbusy(p) == 0) break; - if (!vm_object_page_remove_write(tp, flags, allclean)) { - vm_page_xunbusy(tp); + if (!vm_object_page_remove_write(p, flags, allclean)) { + vm_page_xunbusy(p); break; } - ma[--base] = tp; + ma[--base] = p; } vm_pageout_flush(&ma[base], count, pagerflags, nitems(ma) / 2 - base,