git: 12b613efb4c6 - stable/14 - vm_domainset: Ensure round-robin works properly

The branch stable/14 has been updated by olce:

URL: https://cgit.FreeBSD.org/src/commit/?id=12b613efb4c670959ce540354db353bfb06e72a8

commit 12b613efb4c670959ce540354db353bfb06e72a8
Author:     Olivier Certner <olce@FreeBSD.org>
AuthorDate: 2025-09-25 14:40:30 +0000
Commit:     Olivier Certner <olce@FreeBSD.org>
CommitDate: 2025-12-19 09:16:46 +0000

    vm_domainset: Ensure round-robin works properly
    
    All iterators that rely on an object's 'struct domainset_ref' (field
    'domain' on 'struct vm_object'), which is the case for page allocations
    with objects, are used with the corresponding object locked for writing,
    so cannot lose concurrent iterator index's increases even if those are
    made without atomic operations.  The only offender was thread stack
    allocation, which has just been fixed in commit 3b9b64457676 ("vm: Fix
    iterator usage in vm_thread_stack_create()").
    
    However, the interleaved policy would still reset the iterator index
    when restarting, losing track of the next domain to allocate from when
    applying round-robin, which all allocation policies do if allocation
    from the first domain fails.
    
    Fix this last round-robin problem by not resetting the shared index at
    iterator's phase init on DOMAINSET_POLICY_INTERLEAVE.
    
    Add an assertion to check that, when passed, an object is write-locked
    in order to prevent the problem mentioned in the first paragraph from
    reappearing.
    
    Reviewed by:    markj
    MFC after:      1 week
    Sponsored by:   The FreeBSD Foundation
    Differential Revision:  https://reviews.freebsd.org/D52733
    
    (cherry picked from commit 7b0fe2d405ae09b1247bccc6fa45a6d2755cbe4c)
    
    In stable/14, all page allocations with object have the latter locked
    for writing.  Contrary to what happened in main (and stable/15), the
    offender mentioned in the original commit message never appeared in
    stable/14 because it was introduced in main by commit 7a79d0669761 ("vm:
    improve kstack_object pindex calculation to avoid pindex holes") and
    later fixed by commit 3b9b64457676 ("vm: Fix iterator usage in
    vm_thread_stack_create()"), both of which were never MFCed.  So, the
    following part of the original commit message: "The only offender was
    thread stack allocation, which has just been fixed in commit
    3b9b64457676 ("vm: Fix iterator usage in vm_thread_stack_create()")."
    does not apply here.
---
 sys/vm/vm_domainset.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/sys/vm/vm_domainset.c b/sys/vm/vm_domainset.c
index e10cf2540919..6ec2d893fc7d 100644
--- a/sys/vm/vm_domainset.c
+++ b/sys/vm/vm_domainset.c
@@ -109,7 +109,6 @@ vm_domainset_iter_interleave(struct vm_domainset_iter *di, int *domain)
 	int d;
 
 	d = di->di_offset % di->di_domain->ds_cnt;
-	*di->di_iter = d;
 	*domain = di->di_domain->ds_order[d];
 }
 
@@ -256,9 +255,14 @@ vm_domainset_iter_page_init(struct vm_domainset_iter *di, struct vm_object *obj,
 	 * are immutable and unsynchronized.  Updates can race but pointer
 	 * loads are assumed to be atomic.
 	 */
-	if (obj != NULL && obj->domain.dr_policy != NULL)
+	if (obj != NULL && obj->domain.dr_policy != NULL) {
+		/*
+		 * This write lock protects non-atomic increments of the
+		 * iterator index in vm_domainset_iter_rr().
+		 */
+		VM_OBJECT_ASSERT_WLOCKED(obj);
 		dr = &obj->domain;
-	else
+	} else
 		dr = &curthread->td_domain;
 
 	vm_domainset_iter_init(di, dr->dr_policy, &dr->dr_iter, obj, pindex);