From nobody Tue May 17 12:11:37 2022 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 0D3D91AEE7B1; Tue, 17 May 2022 12:11:40 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4L2ZjQ2MCLz3Q0K; Tue, 17 May 2022 12:11:38 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1652789498; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=hCdHbRf+XpwtCT/+Od0npfAm0yC9b+LagnAInei1KkM=; b=tgRSZwk929uWUf9VzMQRZrAjzesoaNtVmoINrSwY9wXNEAzHN20P1fQHTAVFyVQdpkRWMb ob8lhqISaZJ7zrh7ti/gR9zJQkMYYhMYyLpT3q6I5+n+6D/6jjW1xgl4JZPvS/g88lKo74 8G10JNscy1BpQ7AInQ2/qFCEdAOXV7/2VSD0N/Pkyn/QHqhI98NZhn5wW+b07TysMjBNbg cJNtQwjC5fMayDXf61pGNJhtxyNmGWrv9E67uBZKDF2C3+bkK6Nzidt0pje1XyiSnope9M isYSm9hTyBAvtA4ogBJkEKeEL4bne46ugPhTcQhla2pt9xyvfzdQGCfXJjyv1g== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id E840714840; Tue, 17 May 2022 12:11:37 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 24HCBb3s093205; Tue, 17 May 2022 12:11:37 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 24HCBbFQ093204; Tue, 17 May 2022 12:11:37 GMT (envelope-from git) Date: Tue, 17 May 2022 12:11:37 GMT Message-Id: <202205171211.24HCBbFQ093204@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Vladimir Kondratyev Subject: git: b6f87b78b5bb - main - LinuxKPI: Implement kthread_worker related functions List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-main@freebsd.org X-BeenThere: dev-commits-src-main@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: wulf X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: b6f87b78b5bb48e00f54b96ddea7ad5bf5e3aa1f Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1652789498; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=hCdHbRf+XpwtCT/+Od0npfAm0yC9b+LagnAInei1KkM=; b=i+vlNVX3pHPVr2RU22qWVrbSlzJiqJP8UhWy/FPg8vRxmn4R49vp4Lve0UFunCQrK38iPz uiP8kedD/g1OPfYxjZlHL49MPY85DDRuA4/RZFjevXK4pG6yZ4urHYjfH6ldevR+extPRF 6FDd8uQbArujq0bNZhQkafNX224DQOx36ewM4DKirRzbVl7+UsvH5zgxgX3wQz2HVzaqYf F7SA0vQlr81AU1g6xOR4S25bPYBeNCi8JkLnKmaKEs4+BOr9hsjUBK9l8XJ43PXWZYKcUA bgwlxzFFp5IuKFd9KLz8ECxRzR5H0LoDoa3+bMYOVg0+qEOuIXDjV3nJBM7xyw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1652789498; a=rsa-sha256; cv=none; b=kWEW0yREqwov8S2u/rCezqx15PyPchKe1ACvZWwld+lPZklWvpRrm66ThnOcHmdCF4PqWw HcWHvXWsjAcCl6XwHXVvvUsHxiCTKXmlvJUWvX5JPxDys/uF7dvwe+8P6G+ZbV9pIivfU2 KDipPj7UxhNpJjIfJL2RL9Go3p3MtbYdCu3f3ZpESKmQ9D/GaOYkR4iE2ggeGuemE6NYyS IMa/9PA6VH0SJ4MwF80L4+4h7YjYjroa4bt5i6hoNW4yocMyKpM2M7pMRUq1OQHgT1IIHB ySrez7hJkVsSCgPrOmFwCV4W9XtnEBV00Se+fn1zfw93Q/gn0u4tiULTrtPnpw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by wulf: URL: https://cgit.FreeBSD.org/src/commit/?id=b6f87b78b5bb48e00f54b96ddea7ad5bf5e3aa1f commit b6f87b78b5bb48e00f54b96ddea7ad5bf5e3aa1f Author: Vladimir Kondratyev AuthorDate: 2022-05-17 12:10:20 +0000 Commit: Vladimir Kondratyev CommitDate: 2022-05-17 12:10:20 +0000 LinuxKPI: Implement kthread_worker related functions Kthread worker is a single thread workqueue which can be used in cases where specific kthread association is necessary, for example, when it should have RT priority or be assigned to certain cgroup. This change implements Linux v4.9 interface which mostly hides kthread internals from users thus allowing to use ordinary taskqueue(9) KPI. As kthread worker prohibits enqueueing of already pending or canceling tasks some minimal changes to taskqueue(9) were done. taskqueue_enqueue_flags() was added to taskqueue KPI which accepts extra flags parameter. It contains one or more of the following flags: TASKQUEUE_FAIL_IF_PENDING - taskqueue_enqueue_flags() fails if the task is already scheduled to execution. EEXIST is returned and the ta_pending counter value remains unchanged. TASKQUEUE_FAIL_IF_CANCELING - taskqueue_enqueue_flags() fails if the task is in the canceling state and ECANCELED is returned. Required by: drm-kmod 5.10 MFC after: 1 week Reviewed by: hselasky, Pau Amma (docs) Differential Revision: https://reviews.freebsd.org/D35051 --- share/man/man9/taskqueue.9 | 26 +++++- sys/compat/linuxkpi/common/include/linux/kthread.h | 97 +++++++++++++++++++++- sys/compat/linuxkpi/common/src/linux_kthread.c | 16 ++++ sys/kern/subr_taskqueue.c | 74 ++++++++++++----- sys/sys/taskqueue.h | 6 ++ 5 files changed, 196 insertions(+), 23 deletions(-) diff --git a/share/man/man9/taskqueue.9 b/share/man/man9/taskqueue.9 index 58eb6b7c5571..e13ad9498562 100644 --- a/share/man/man9/taskqueue.9 +++ b/share/man/man9/taskqueue.9 @@ -28,7 +28,7 @@ .\" .\" $FreeBSD$ .\" -.Dd September 1, 2021 +.Dd April 25, 2022 .Dt TASKQUEUE 9 .Os .Sh NAME @@ -85,6 +85,8 @@ struct timeout_task; .Ft int .Fn taskqueue_enqueue "struct taskqueue *queue" "struct task *task" .Ft int +.Fn taskqueue_enqueue_flags "struct taskqueue *queue" "struct task *task" "int flags" +.Ft int .Fn taskqueue_enqueue_timeout "struct taskqueue *queue" "struct timeout_task *timeout_task" "int ticks" .Ft int .Fn taskqueue_enqueue_timeout_sbt "struct taskqueue *queue" "struct timeout_task *timeout_task" "sbintime_t sbt" "sbintime_t pr" "int flags" @@ -225,6 +227,28 @@ is called on the task pointer passed to .Fn taskqueue_enqueue . .Pp The +.Fn taskqueue_enqueue_flags +accepts an extra +.Va flags +parameter which specifies a set of optional flags to alter the behavior of +.Fn taskqueue_enqueue . +It contains one or more of the following flags: +.Bl -tag -width TASKQUEUE_FAIL_IF_CANCELING +.It Dv TASKQUEUE_FAIL_IF_PENDING +.Fn taskqueue_enqueue_flags +fails if the task is already scheduled for execution. +.Er EEXIST +is returned and the +.Va ta_pending +counter value remains unchanged. +.It Dv TASKQUEUE_FAIL_IF_CANCELING +.Fn taskqueue_enqueue_flags +fails if the task is in the canceling state and +.Er ECANCELED +is returned. +.El +.Pp +The .Fn taskqueue_enqueue_timeout function is used to schedule the enqueue after the specified number of .Va ticks . diff --git a/sys/compat/linuxkpi/common/include/linux/kthread.h b/sys/compat/linuxkpi/common/include/linux/kthread.h index f4791d5f0db3..49309cd47a40 100644 --- a/sys/compat/linuxkpi/common/include/linux/kthread.h +++ b/sys/compat/linuxkpi/common/include/linux/kthread.h @@ -33,8 +33,29 @@ #include -#include +#include +#include #include +#include +#include +#include +#include + +struct task_struct; +struct kthread_work; + +typedef void (*kthread_work_func_t)(struct kthread_work *work); + +struct kthread_worker { + struct task_struct *task; + struct taskqueue *tq; +}; + +struct kthread_work { + struct taskqueue *tq; + struct task task; + kthread_work_func_t func; +}; #define kthread_run(fn, data, fmt, ...) ({ \ struct task_struct *__task; \ @@ -70,4 +91,78 @@ int linux_in_atomic(void); #define in_atomic() linux_in_atomic() +/* Only kthread_(create|destroy)_worker interface is allowed */ +#define kthread_init_worker(worker) \ + _Static_assert(false, "pre-4.9 worker interface is not supported"); + +task_fn_t lkpi_kthread_work_fn; +task_fn_t lkpi_kthread_worker_init_fn; + +#define kthread_create_worker(flags, fmt, ...) ({ \ + struct kthread_worker *__w; \ + struct task __task; \ + \ + __w = malloc(sizeof(*__w), M_KMALLOC, M_WAITOK | M_ZERO); \ + __w->tq = taskqueue_create("lkpi kthread taskq", M_WAITOK, \ + taskqueue_thread_enqueue, &__w->tq); \ + taskqueue_start_threads(&__w->tq, 1, PWAIT, fmt, ##__VA_ARGS__);\ + TASK_INIT(&__task, 0, lkpi_kthread_worker_init_fn, __w); \ + taskqueue_enqueue(__w->tq, &__task); \ + taskqueue_drain(__w->tq, &__task); \ + __w; \ +}) + +static inline void +kthread_destroy_worker(struct kthread_worker *worker) +{ + taskqueue_drain_all(worker->tq); + taskqueue_free(worker->tq); + free(worker, M_KMALLOC); +} + +static inline void +kthread_init_work(struct kthread_work *work, kthread_work_func_t func) +{ + work->tq = NULL; + work->func = func; + TASK_INIT(&work->task, 0, lkpi_kthread_work_fn, work); +} + +static inline bool +kthread_queue_work(struct kthread_worker *worker, struct kthread_work *work) +{ + int error; + + error = taskqueue_enqueue_flags(worker->tq, &work->task, + TASKQUEUE_FAIL_IF_CANCELING | TASKQUEUE_FAIL_IF_PENDING); + if (error == 0) + work->tq = worker->tq; + return (error == 0); +} + +static inline bool +kthread_cancel_work_sync(struct kthread_work *work) +{ + u_int pending = 0; + + if (work->tq != NULL && + taskqueue_cancel(work->tq, &work->task, &pending) != 0) + taskqueue_drain(work->tq, &work->task); + + return (pending != 0); +} + +static inline void +kthread_flush_work(struct kthread_work *work) +{ + if (work->tq != NULL) + taskqueue_drain(work->tq, &work->task); +} + +static inline void +kthread_flush_worker(struct kthread_worker *worker) +{ + taskqueue_drain_all(worker->tq); +} + #endif /* _LINUXKPI_LINUX_KTHREAD_H_ */ diff --git a/sys/compat/linuxkpi/common/src/linux_kthread.c b/sys/compat/linuxkpi/common/src/linux_kthread.c index 26afe005ea59..19afad6872c3 100644 --- a/sys/compat/linuxkpi/common/src/linux_kthread.c +++ b/sys/compat/linuxkpi/common/src/linux_kthread.c @@ -165,3 +165,19 @@ linux_kthread_fn(void *arg __unused) } kthread_exit(); } + +void +lkpi_kthread_work_fn(void *context, int pending __unused) +{ + struct kthread_work *work = context; + + work->func(work); +} + +void +lkpi_kthread_worker_init_fn(void *context, int pending __unused) +{ + struct kthread_worker *worker = context; + + worker->task = current; +} diff --git a/sys/kern/subr_taskqueue.c b/sys/kern/subr_taskqueue.c index e43b09010761..7ad7c210ceff 100644 --- a/sys/kern/subr_taskqueue.c +++ b/sys/kern/subr_taskqueue.c @@ -59,6 +59,7 @@ static void taskqueue_swi_giant_enqueue(void *); struct taskqueue_busy { struct task *tb_running; u_int tb_seq; + bool tb_canceling; LIST_ENTRY(taskqueue_busy) tb_link; }; @@ -125,6 +126,19 @@ TQ_SLEEP(struct taskqueue *tq, void *p, const char *wm) return (msleep(p, &tq->tq_mutex, 0, wm, 0)); } +static struct taskqueue_busy * +task_get_busy(struct taskqueue *queue, struct task *task) +{ + struct taskqueue_busy *tb; + + TQ_ASSERT_LOCKED(queue); + LIST_FOREACH(tb, &queue->tq_active, tb_link) { + if (tb->tb_running == task) + return (tb); + } + return (NULL); +} + static struct taskqueue * _taskqueue_create(const char *name, int mflags, taskqueue_enqueue_fn enqueue, void *context, @@ -217,16 +231,32 @@ taskqueue_free(struct taskqueue *queue) } static int -taskqueue_enqueue_locked(struct taskqueue *queue, struct task *task) +taskqueue_enqueue_locked(struct taskqueue *queue, struct task *task, int flags) { struct task *ins; struct task *prev; + struct taskqueue_busy *tb; KASSERT(task->ta_func != NULL, ("enqueueing task with NULL func")); + /* + * Ignore canceling task if requested. + */ + if (__predict_false((flags & TASKQUEUE_FAIL_IF_CANCELING) != 0)) { + tb = task_get_busy(queue, task); + if (tb != NULL && tb->tb_canceling) { + TQ_UNLOCK(queue); + return (ECANCELED); + } + } + /* * Count multiple enqueues. */ if (task->ta_pending) { + if (__predict_false((flags & TASKQUEUE_FAIL_IF_PENDING) != 0)) { + TQ_UNLOCK(queue); + return (EEXIST); + } if (task->ta_pending < USHRT_MAX) task->ta_pending++; TQ_UNLOCK(queue); @@ -274,17 +304,23 @@ taskqueue_enqueue_locked(struct taskqueue *queue, struct task *task) } int -taskqueue_enqueue(struct taskqueue *queue, struct task *task) +taskqueue_enqueue_flags(struct taskqueue *queue, struct task *task, int flags) { int res; TQ_LOCK(queue); - res = taskqueue_enqueue_locked(queue, task); + res = taskqueue_enqueue_locked(queue, task, flags); /* The lock is released inside. */ return (res); } +int +taskqueue_enqueue(struct taskqueue *queue, struct task *task) +{ + return (taskqueue_enqueue_flags(queue, task, 0)); +} + static void taskqueue_timeout_func(void *arg) { @@ -296,7 +332,7 @@ taskqueue_timeout_func(void *arg) KASSERT((timeout_task->f & DT_CALLOUT_ARMED) != 0, ("Stray timeout")); timeout_task->f &= ~DT_CALLOUT_ARMED; queue->tq_callouts--; - taskqueue_enqueue_locked(timeout_task->q, &timeout_task->t); + taskqueue_enqueue_locked(timeout_task->q, &timeout_task->t, 0); /* The lock is released inside. */ } @@ -316,7 +352,7 @@ taskqueue_enqueue_timeout_sbt(struct taskqueue *queue, TQ_UNLOCK(queue); res = -1; } else if (sbt == 0) { - taskqueue_enqueue_locked(queue, &timeout_task->t); + taskqueue_enqueue_locked(queue, &timeout_task->t, 0); /* The lock is released inside. */ } else { if ((timeout_task->f & DT_CALLOUT_ARMED) != 0) { @@ -464,6 +500,7 @@ taskqueue_run_locked(struct taskqueue *queue) task->ta_pending = 0; tb.tb_running = task; tb.tb_seq = ++queue->tq_seq; + tb.tb_canceling = false; TQ_UNLOCK(queue); KASSERT(task->ta_func != NULL, ("task->ta_func is NULL")); @@ -493,19 +530,6 @@ taskqueue_run(struct taskqueue *queue) TQ_UNLOCK(queue); } -static int -task_is_running(struct taskqueue *queue, struct task *task) -{ - struct taskqueue_busy *tb; - - TQ_ASSERT_LOCKED(queue); - LIST_FOREACH(tb, &queue->tq_active, tb_link) { - if (tb->tb_running == task) - return (1); - } - return (0); -} - /* * Only use this function in single threaded contexts. It returns * non-zero if the given task is either pending or running. Else the @@ -517,7 +541,7 @@ taskqueue_poll_is_busy(struct taskqueue *queue, struct task *task) int retval; TQ_LOCK(queue); - retval = task->ta_pending > 0 || task_is_running(queue, task); + retval = task->ta_pending > 0 || task_get_busy(queue, task) != NULL; TQ_UNLOCK(queue); return (retval); @@ -527,6 +551,8 @@ static int taskqueue_cancel_locked(struct taskqueue *queue, struct task *task, u_int *pendp) { + struct taskqueue_busy *tb; + int retval = 0; if (task->ta_pending > 0) { STAILQ_REMOVE(&queue->tq_queue, task, task, ta_link); @@ -536,7 +562,13 @@ taskqueue_cancel_locked(struct taskqueue *queue, struct task *task, if (pendp != NULL) *pendp = task->ta_pending; task->ta_pending = 0; - return (task_is_running(queue, task) ? EBUSY : 0); + tb = task_get_busy(queue, task); + if (tb != NULL) { + tb->tb_canceling = true; + retval = EBUSY; + } + + return (retval); } int @@ -580,7 +612,7 @@ taskqueue_drain(struct taskqueue *queue, struct task *task) WITNESS_WARN(WARN_GIANTOK | WARN_SLEEPOK, NULL, __func__); TQ_LOCK(queue); - while (task->ta_pending != 0 || task_is_running(queue, task)) + while (task->ta_pending != 0 || task_get_busy(queue, task) != NULL) TQ_SLEEP(queue, task, "tq_drain"); TQ_UNLOCK(queue); } diff --git a/sys/sys/taskqueue.h b/sys/sys/taskqueue.h index 7e59187e0114..cc98fe5fab85 100644 --- a/sys/sys/taskqueue.h +++ b/sys/sys/taskqueue.h @@ -61,6 +61,10 @@ enum taskqueue_callback_type { #define TASKQUEUE_NUM_CALLBACKS TASKQUEUE_CALLBACK_TYPE_MAX + 1 #define TASKQUEUE_NAMELEN 32 +/* taskqueue_enqueue flags */ +#define TASKQUEUE_FAIL_IF_PENDING (1 << 0) +#define TASKQUEUE_FAIL_IF_CANCELING (1 << 1) + typedef void (*taskqueue_callback_fn)(void *context); /* @@ -82,6 +86,8 @@ int taskqueue_start_threads_in_proc(struct taskqueue **tqp, int count, int taskqueue_start_threads_cpuset(struct taskqueue **tqp, int count, int pri, cpuset_t *mask, const char *name, ...) __printflike(5, 6); int taskqueue_enqueue(struct taskqueue *queue, struct task *task); +int taskqueue_enqueue_flags(struct taskqueue *queue, struct task *task, + int flags); int taskqueue_enqueue_timeout(struct taskqueue *queue, struct timeout_task *timeout_task, int ticks); int taskqueue_enqueue_timeout_sbt(struct taskqueue *queue,