git: 7bbac6419d17 - main - Schedule fast taskqueue callouts on right CPU.

The branch main has been updated by mav:

URL: https://cgit.FreeBSD.org/src/commit/?id=7bbac6419d174c98cc6ea969b68fcfe0f9a9bab8

commit 7bbac6419d174c98cc6ea969b68fcfe0f9a9bab8
Author:     Alexander Motin <mav@FreeBSD.org>
AuthorDate: 2023-12-27 03:30:56 +0000
Commit:     Alexander Motin <mav@FreeBSD.org>
CommitDate: 2023-12-27 03:55:24 +0000

    Schedule fast taskqueue callouts on right CPU.
    
    With fast taskqueues using direct callouts we can reduce number of
    CPU wakeups by scheduling callout on current CPU if taskqueue calls
    taskqueue_enqueue_timeout() on itself.  The trick won't work for
    regular taskqueues, since the callout thread will occupy the CPU.
    It also may not work in case of multiple threads since we do not
    know which thread will pick the task, and we do not want excessive
    callout migrations.  So we optimize only the other cases we can.
    
    In practice this allows iichid(4) taskqueue to stay on CPU where
    underlying ig4(4) interrupts are routed and to not kick CPU 0 with
    timer interrupts on each sampling period (every 2nd/3rd sleep).
    
    MFC after:      1 month
---
 sys/kern/subr_taskqueue.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/sys/kern/subr_taskqueue.c b/sys/kern/subr_taskqueue.c
index 5650f9d57a01..c4146922edc5 100644
--- a/sys/kern/subr_taskqueue.c
+++ b/sys/kern/subr_taskqueue.c
@@ -363,8 +363,14 @@ taskqueue_enqueue_timeout_sbt(struct taskqueue *queue,
 		if (sbt > 0) {
 			if (queue->tq_spin)
 				flags |= C_DIRECT_EXEC;
-			callout_reset_sbt(&timeout_task->c, sbt, pr,
-			    taskqueue_timeout_func, timeout_task, flags);
+			if (queue->tq_spin && queue->tq_tcount == 1 &&
+			    queue->tq_threads[0] == curthread) {
+				callout_reset_sbt_curcpu(&timeout_task->c, sbt, pr,
+				    taskqueue_timeout_func, timeout_task, flags);
+			} else {
+				callout_reset_sbt(&timeout_task->c, sbt, pr,
+				    taskqueue_timeout_func, timeout_task, flags);
+			}
 		}
 		TQ_UNLOCK(queue);
 	}