From nobody Mon Jul 28 13:31:14 2025 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4brKBC5xvbz634n6; Mon, 28 Jul 2025 13:31:15 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4brKBB6zqLz3tbW; Mon, 28 Jul 2025 13:31:14 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1753709475; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=lSRSvDh+GpcCsD53ecBMDmjYQMB9sB31Qil6FQPT1hE=; b=JsohuKHw77q9P6AUO2q+rKejMxEEI2i1tD7Z9CRK3iFUh/70zQkvw1GNfJx+2ULVRIN98+ FHpVbFmv1S9Qz1YwSVB8uPOKxwxL7ogfYxko/fCG45zsy3rriPl8t67RELSkUiLaHwtkCM KDZiki3k6KkEr7lwlJ3DTKTSKVtL6fZ2YgLCwpdau1EShIJYFfhptyf7SACv4nxUSSuj4K KX4aUWGGBNlIXX9ncuBFXUFeR+LNPf2zmqeqGAS73E8oIMMGpE4Kdea2lYtZ5bDJUH2Xvd pnY0irmkOWh+yDQgfJ+Wi9sohynmdz5lRPbmesuRgKojvJY0VsCYzpI8/nF7lA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1753709475; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=lSRSvDh+GpcCsD53ecBMDmjYQMB9sB31Qil6FQPT1hE=; b=TsATvKcykVnmkjNEv9O/NC2zpneu5r7zIYXly/mgpPg8ARPXz7UW5BUYMmmxH0WJrbH0RW CbNkbJgrNeuQ4kDx2oYfOQGauX6YNpWLaUjcVG5yF390pWhGq4U7MK3q3l91hlzpdfI+fR BFxurS3oQPWf6N1ImQqsjI5zrtZ6GriYy2Jv03f/KEETa4ep9dDjQcLDNjXMbnh9iTx82m cWuttHuCjO840pWdpTqpevXY3zx2dcuoChKEfEf/VEJJ45PdPcPS5HWVTcR3o1mfxvCoz7 WA3Ks/CLqrhGqIZKDgN1JrseTE5RgH4u18uLO6RHEyFPWlkcOLAQQ6bsltlanw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1753709475; a=rsa-sha256; cv=none; b=fPcEwL6IoTuTWFLhkxBaZNcWso87b9WGyMW/xjDcAgziKhACt8jZj9wJn7JpsDapoDRQcq /0WtgOSlgrTDFP8sZbupuhXWVwtYyttpNuchp5+kW4XyzH8dzpSSiU5hBg1YM4REytX0IM hrbgO4IGexoebxD6iwU4Ra3RCNxAay0n1XzCe6Lj8YpvEQ7BCrcO392LBPaL2oVdb3PIh+ WIPDkm3IWSd8KPbLlV5ioCBjw+0COCjZaRp480Jx+cBRYNcLjFCd42r+Or4Afh6mUO4vIy zOcPhYRUpIONmEahrmw65STPkH9ebS8hr8RxO0ocvLG2AQq5L88ufrgNkxAF4g== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4brKBB6YKLznfp; Mon, 28 Jul 2025 13:31:14 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 56SDVE8U080089; Mon, 28 Jul 2025 13:31:14 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 56SDVEqN080086; Mon, 28 Jul 2025 13:31:14 GMT (envelope-from git) Date: Mon, 28 Jul 2025 13:31:14 GMT Message-Id: <202507281331.56SDVEqN080086@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Olivier Certner Subject: git: 606087082472 - stable/14 - runq: Re-order functions more logically List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: olce X-Git-Repository: src X-Git-Refname: refs/heads/stable/14 X-Git-Reftype: branch X-Git-Commit: 60608708247207a69d3ec88e4cb05b066d3f3b74 Auto-Submitted: auto-generated The branch stable/14 has been updated by olce: URL: https://cgit.FreeBSD.org/src/commit/?id=60608708247207a69d3ec88e4cb05b066d3f3b74 commit 60608708247207a69d3ec88e4cb05b066d3f3b74 Author: Olivier Certner AuthorDate: 2024-03-01 13:26:39 +0000 Commit: Olivier Certner CommitDate: 2025-07-28 13:28:07 +0000 runq: Re-order functions more logically No code change in moved functions. Reviewed by: kib MFC after: 1 month Event: Kitchener-Waterloo Hackathon 202506 Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D45387 (cherry picked from commit 200fc93dace76cf251460adc58ca809c36031f3e) --- sys/kern/kern_switch.c | 201 +++++++++++++++++++++++++------------------------ sys/sys/runq.h | 7 +- 2 files changed, 105 insertions(+), 103 deletions(-) diff --git a/sys/kern/kern_switch.c b/sys/kern/kern_switch.c index 9e63709d0acc..69e6f4818e40 100644 --- a/sys/kern/kern_switch.c +++ b/sys/kern/kern_switch.c @@ -274,6 +274,24 @@ runq_init(struct runq *rq) TAILQ_INIT(&rq->rq_queues[i]); } +/* + * Set the status bit of the queue at index 'idx', indicating that it is + * non-empty. + */ +static __inline void +runq_setbit(struct runq *rq, int idx) +{ + struct rq_status *rqs; + + CHECK_IDX(idx); + rqs = &rq->rq_status; + CTR4(KTR_RUNQ, "runq_setbit: bits=%#x %#x bit=%#x word=%d", + rqs->rq_sw[RQSW_IDX(idx)], + rqs->rq_sw[RQSW_IDX(idx)] | RQSW_BIT(idx), + RQSW_BIT(idx), RQSW_IDX(idx)); + rqs->rq_sw[RQSW_IDX(idx)] |= RQSW_BIT(idx); +} + /* * Clear the status bit of the queue at index 'idx', indicating that it is * empty. @@ -292,6 +310,65 @@ runq_clrbit(struct runq *rq, int idx) rqs->rq_sw[RQSW_IDX(idx)] &= ~RQSW_BIT(idx); } +/* + * Add the thread to the queue specified by its priority, and set the + * corresponding status bit. + */ +void +runq_add(struct runq *rq, struct thread *td, int flags) +{ + + runq_add_idx(rq, td, RQ_PRI_TO_QUEUE_IDX(td->td_priority), flags); +} + +void +runq_add_idx(struct runq *rq, struct thread *td, int idx, int flags) +{ + struct rq_queue *rqq; + + /* + * runq_setbit() asserts 'idx' is non-negative and below 'RQ_NQS', and + * a static assert earlier in this file ensures that 'RQ_NQS' is no more + * than 256. + */ + td->td_rqindex = idx; + runq_setbit(rq, idx); + rqq = &rq->rq_queues[idx]; + CTR4(KTR_RUNQ, "runq_add_idx: td=%p pri=%d idx=%d rqq=%p", + td, td->td_priority, idx, rqq); + if (flags & SRQ_PREEMPTED) + TAILQ_INSERT_HEAD(rqq, td, td_runq); + else + TAILQ_INSERT_TAIL(rqq, td, td_runq); +} + +/* + * Remove the thread from the queue specified by its priority, and clear the + * corresponding status bit if the queue becomes empty. + * + * Returns whether the corresponding queue is empty after removal. + */ +bool +runq_remove(struct runq *rq, struct thread *td) +{ + struct rq_queue *rqq; + int idx; + + KASSERT(td->td_flags & TDF_INMEM, ("runq_remove: Thread swapped out")); + idx = td->td_rqindex; + CHECK_IDX(idx); + rqq = &rq->rq_queues[idx]; + CTR4(KTR_RUNQ, "runq_remove: td=%p pri=%d idx=%d rqq=%p", + td, td->td_priority, idx, rqq); + TAILQ_REMOVE(rqq, td, td_runq); + if (TAILQ_EMPTY(rqq)) { + runq_clrbit(rq, idx); + CTR1(KTR_RUNQ, "runq_remove: queue at idx=%d now empty", idx); + return (true); + } + return (false); +} + /* * Find the index of the first non-empty run queue. This is done by * scanning the status bits, a set bit indicating a non-empty queue. @@ -346,56 +423,6 @@ again: goto again; } -/* - * Set the status bit of the queue at index 'idx', indicating that it is - * non-empty. - */ -static __inline void -runq_setbit(struct runq *rq, int idx) -{ - struct rq_status *rqs; - - CHECK_IDX(idx); - rqs = &rq->rq_status; - CTR4(KTR_RUNQ, "runq_setbit: bits=%#x %#x bit=%#x word=%d", - rqs->rq_sw[RQSW_IDX(idx)], - rqs->rq_sw[RQSW_IDX(idx)] | RQSW_BIT(idx), - RQSW_BIT(idx), RQSW_IDX(idx)); - rqs->rq_sw[RQSW_IDX(idx)] |= RQSW_BIT(idx); -} - -/* - * Add the thread to the queue specified by its priority, and set the - * corresponding status bit. - */ -void -runq_add(struct runq *rq, struct thread *td, int flags) -{ - - runq_add_idx(rq, td, RQ_PRI_TO_QUEUE_IDX(td->td_priority), flags); -} - -void -runq_add_idx(struct runq *rq, struct thread *td, int idx, int flags) -{ - struct rq_queue *rqq; - - /* - * runq_setbit() asserts 'idx' is non-negative and below 'RQ_NQS', and - * a static assert earlier in this file ensures that 'RQ_NQS' is no more - * than 256. - */ - td->td_rqindex = idx; - runq_setbit(rq, idx); - rqq = &rq->rq_queues[idx]; - CTR4(KTR_RUNQ, "runq_add_idx: td=%p pri=%d idx=%d rqq=%p", - td, td->td_priority, idx, rqq); - if (flags & SRQ_PREEMPTED) - TAILQ_INSERT_HEAD(rqq, td, td_runq); - else - TAILQ_INSERT_TAIL(rqq, td, td_runq); -} - /* * Return true if there are some processes of any priority on the run queue, * false otherwise. Has no side effects. @@ -417,6 +444,30 @@ runq_not_empty(struct runq *rq) return (false); } +/* + * Find the highest priority process on the run queue. + */ +struct thread * +runq_choose(struct runq *rq) +{ + struct rq_queue *rqq; + struct thread *td; + int idx; + + idx = runq_findbit(rq); + if (idx != -1) { + rqq = &rq->rq_queues[idx]; + td = TAILQ_FIRST(rqq); + KASSERT(td != NULL, ("runq_choose: no thread on busy queue")); + CTR3(KTR_RUNQ, + "runq_choose: idx=%d thread=%p rqq=%p", idx, td, rqq); + return (td); + } + CTR1(KTR_RUNQ, "runq_choose: idlethread idx=%d", idx); + + return (NULL); +} + /* * Find the highest priority process on the run queue. */ @@ -460,30 +511,6 @@ runq_choose_fuzz(struct runq *rq, int fuzz) return (NULL); } -/* - * Find the highest priority process on the run queue. - */ -struct thread * -runq_choose(struct runq *rq) -{ - struct rq_queue *rqq; - struct thread *td; - int idx; - - idx = runq_findbit(rq); - if (idx != -1) { - rqq = &rq->rq_queues[idx]; - td = TAILQ_FIRST(rqq); - KASSERT(td != NULL, ("runq_choose: no thread on busy queue")); - CTR3(KTR_RUNQ, - "runq_choose: idx=%d thread=%p rqq=%p", idx, td, rqq); - return (td); - } - CTR1(KTR_RUNQ, "runq_choose: idlethread idx=%d", idx); - - return (NULL); -} - struct thread * runq_choose_from(struct runq *rq, int from_idx) { @@ -504,29 +531,3 @@ runq_choose_from(struct runq *rq, int from_idx) return (NULL); } -/* - * Remove the thread from the queue specified by its priority, and clear the - * corresponding status bit if the queue becomes empty. - * - * Returns whether the corresponding queue is empty after removal. - */ -bool -runq_remove(struct runq *rq, struct thread *td) -{ - struct rq_queue *rqq; - int idx; - - KASSERT(td->td_flags & TDF_INMEM, ("runq_remove: Thread swapped out")); - idx = td->td_rqindex; - CHECK_IDX(idx); - rqq = &rq->rq_queues[idx]; - CTR4(KTR_RUNQ, "runq_remove: td=%p pri=%d idx=%d rqq=%p", - td, td->td_priority, idx, rqq); - TAILQ_REMOVE(rqq, td, td_runq); - if (TAILQ_EMPTY(rqq)) { - runq_clrbit(rq, idx); - CTR1(KTR_RUNQ, "runq_remove: queue at idx=%d now empty", idx); - return (true); - } - return (false); -} diff --git a/sys/sys/runq.h b/sys/sys/runq.h index 80a1fad0a089..c570dd25503b 100644 --- a/sys/sys/runq.h +++ b/sys/sys/runq.h @@ -97,14 +97,15 @@ struct runq { struct rq_queue rq_queues[RQ_NQS]; }; +void runq_init(struct runq *); void runq_add(struct runq *, struct thread *, int _flags); void runq_add_idx(struct runq *, struct thread *, int _idx, int _flags); +bool runq_remove(struct runq *, struct thread *); + bool runq_not_empty(struct runq *); struct thread *runq_choose(struct runq *); -struct thread *runq_choose_from(struct runq *, int _idx); struct thread *runq_choose_fuzz(struct runq *, int _fuzz); -void runq_init(struct runq *); -bool runq_remove(struct runq *, struct thread *); +struct thread *runq_choose_from(struct runq *, int _idx); #endif /* _KERNEL */ #endif