From nobody Wed Feb 02 00:53:21 2022 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4613719A1E69; Wed, 2 Feb 2022 00:53:22 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4JpNZp1NLNz59DN; Wed, 2 Feb 2022 00:53:22 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1643763202; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=9aiKCZwrGdfMiSLkB4Hv+1Sk8Mit/LMYcgsl2Q9iMMY=; b=FTlYi03unDB9Sn7J4zLt3EOT64o8ZWqmDUXVcCWXR/TGDMROO3H7XlvJn5V2iokCo5ftjv ixOL2xJ3HZ1iFZ7x4GOZS04a83ML4ABgXjlv4RzCtUH/ImAlj8Z9rj9X4VFkYEL4nJc9Nz PR1LcoVh0Oi6jpvVIZTekDF6rgb2mzbmR7X3cNhR5MdctDIE2N67oQUrpyXuFlFpZ8nvVC d2ZM1AyeAyYUM1yzTZ/eDjDniFtW+sgB2T0TkZyTkpOp/xO8+QzSGWfQzbQLDubK5Z81lP VllwEXZXbzaqKD2kWH/9D5oPTELrSpkImywV4kK72IJgZC4WNTU1Nv2Fxg9KRw== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 109C91ADB4; Wed, 2 Feb 2022 00:53:22 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 2120rLPT015236; Wed, 2 Feb 2022 00:53:21 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 2120rLM0015235; Wed, 2 Feb 2022 00:53:21 GMT (envelope-from git) Date: Wed, 2 Feb 2022 00:53:21 GMT Message-Id: <202202020053.2120rLM0015235@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Alexander Motin Subject: git: c27237d62f05 - stable/13 - Reduce bufdaemon/bufspacedaemon shutdown time. List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mav X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: c27237d62f05f3a1842818c5f3cf87fb861a8402 Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1643763202; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=9aiKCZwrGdfMiSLkB4Hv+1Sk8Mit/LMYcgsl2Q9iMMY=; b=jQQSOXDZf/tx8wGT7uWNqGCwZMW3+Z1UdTvC0JkFpgKHgpeo2PIhPrkuOI+2nwaJgb11Tb NtC4fGsC/aRPRxzbAEWsoQVbq0Gs0jh0Qw8KnaDprNctLsIGyt6QhGk/h9BjFdCkbB//zj bKCzqRR6ayoqFzI++zHcVxSdtO+D2Oc3LZNSrjqB1r9iyZFoHTTigBmurMuYb8mLPv23zT OprYKwZPAePpntIjnpn8Mz45FNfGIC+Jhz1UCwRaqJfr1n5a1yoc+0d7Mb0R+NpI+lQigA BpFIXf5jHKhk3H24TUuK0+ZDyCBtEFjgz0LijuGV4Q7K2R4Krs+Rzkywlv1xrg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1643763202; a=rsa-sha256; cv=none; b=ECDTwqLleDO2ByLJSGCzttHDW7YYxzbmCm3ZYeESNT3yEri6lYDDuDPR+Inrt4HB1uglxr 3y7nLds5ubjdQwCgJ12YDffgfhdh9Xnpde8djYUnn4oYB0eKWP5TcBi2n+LauhKxDpECgb 2egPEqJuwQ4gHbeCW0c3gWyWDNVIib4QIvFuQn3aZdS18LWAkHsT63ayp7kNuVULwd7ut2 5QurpdEr0dcMOXAov8FXP1aczPJDTtjEvyX5EnaIeqSCdcS00b/QKHXRkRB8fLo+mo24vP jqhMyucGnpqV4LVLFkYZxue1VufQE4D2SGBcx5wIsENb/WaANUA8MOtEGrSfBg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch stable/13 has been updated by mav: URL: https://cgit.FreeBSD.org/src/commit/?id=c27237d62f05f3a1842818c5f3cf87fb861a8402 commit c27237d62f05f3a1842818c5f3cf87fb861a8402 Author: Alexander Motin AuthorDate: 2022-01-19 00:26:16 +0000 Commit: Alexander Motin CommitDate: 2022-02-02 00:53:10 +0000 Reduce bufdaemon/bufspacedaemon shutdown time. Before this change bufdaemon and bufspacedaemon threads used kthread_shutdown() to stop activity on system shutdown. The problem is that kthread_shutdown() has no idea about the wait channel and lock used by specific thread to wake them up reliably. As result, up to 9 threads could consume up to 9 seconds to shutdown for no good reason. This change introduces specific shutdown functions, knowing how to properly wake up specific threads, reducing wait for those threads on shutdown/reboot from average 4 seconds to effectively zero. MFC after: 2 weeks Reviewed by: kib, markj Differential Revision: https://reviews.freebsd.org/D33936 (cherry picked from commit b7ff445ffa38282daeab36ce82681ba3f54c8851) --- sys/kern/vfs_bio.c | 108 +++++++++++++++++++++++++++++++++++------------------ 1 file changed, 71 insertions(+), 37 deletions(-) diff --git a/sys/kern/vfs_bio.c b/sys/kern/vfs_bio.c index 5c5cfc8cd5d5..a3cec5d7ec74 100644 --- a/sys/kern/vfs_bio.c +++ b/sys/kern/vfs_bio.c @@ -133,6 +133,7 @@ struct bufdomain { int bd_lim; /* atomics */ int bd_wanted; + bool bd_shutdown; int __aligned(CACHE_LINE_SIZE) bd_numdirtybuffers; int __aligned(CACHE_LINE_SIZE) bd_running; long __aligned(CACHE_LINE_SIZE) bd_bufspace; @@ -340,6 +341,11 @@ static struct mtx_padalign __exclusive_cache_line rbreqlock; */ static struct mtx_padalign __exclusive_cache_line bdirtylock; +/* + * bufdaemon shutdown request and sleep channel. + */ +static bool bd_shutdown; + /* * Wakeup point for bufdaemon, as well as indicator of whether it is already * active. Set to 1 when the bufdaemon is already "on" the queue, 0 when it @@ -628,33 +634,6 @@ bufspace_daemon_wakeup(struct bufdomain *bd) } } -/* - * bufspace_daemon_wait: - * - * Sleep until the domain falls below a limit or one second passes. - */ -static void -bufspace_daemon_wait(struct bufdomain *bd) -{ - /* - * Re-check our limits and sleep. bd_running must be - * cleared prior to checking the limits to avoid missed - * wakeups. The waker will adjust one of bufspace or - * freebuffers prior to checking bd_running. - */ - BD_RUN_LOCK(bd); - atomic_store_int(&bd->bd_running, 0); - if (bd->bd_bufspace < bd->bd_bufspacethresh && - bd->bd_freebuffers > bd->bd_lofreebuffers) { - msleep(&bd->bd_running, BD_RUN_LOCKPTR(bd), PRIBIO|PDROP, - "-", hz); - } else { - /* Avoid spurious wakeups while running. */ - atomic_store_int(&bd->bd_running, 1); - BD_RUN_UNLOCK(bd); - } -} - /* * bufspace_adjust: * @@ -785,6 +764,22 @@ bufspace_wait(struct bufdomain *bd, struct vnode *vp, int gbflags, BD_UNLOCK(bd); } +static void +bufspace_daemon_shutdown(void *arg, int howto __unused) +{ + struct bufdomain *bd = arg; + int error; + + BD_RUN_LOCK(bd); + bd->bd_shutdown = true; + wakeup(&bd->bd_running); + error = msleep(&bd->bd_shutdown, BD_RUN_LOCKPTR(bd), 0, + "bufspace_shutdown", 60 * hz); + BD_RUN_UNLOCK(bd); + if (error != 0) + printf("bufspacedaemon wait error: %d\n", error); +} + /* * bufspace_daemon: * @@ -795,14 +790,14 @@ bufspace_wait(struct bufdomain *bd, struct vnode *vp, int gbflags, static void bufspace_daemon(void *arg) { - struct bufdomain *bd; + struct bufdomain *bd = arg; - EVENTHANDLER_REGISTER(shutdown_pre_sync, kthread_shutdown, curthread, + EVENTHANDLER_REGISTER(shutdown_pre_sync, bufspace_daemon_shutdown, bd, SHUTDOWN_PRI_LAST + 100); - bd = arg; - for (;;) { - kthread_suspend_check(); + BD_RUN_LOCK(bd); + while (!bd->bd_shutdown) { + BD_RUN_UNLOCK(bd); /* * Free buffers from the clean queue until we meet our @@ -852,8 +847,29 @@ bufspace_daemon(void *arg) } maybe_yield(); } - bufspace_daemon_wait(bd); + + /* + * Re-check our limits and sleep. bd_running must be + * cleared prior to checking the limits to avoid missed + * wakeups. The waker will adjust one of bufspace or + * freebuffers prior to checking bd_running. + */ + BD_RUN_LOCK(bd); + if (bd->bd_shutdown) + break; + atomic_store_int(&bd->bd_running, 0); + if (bd->bd_bufspace < bd->bd_bufspacethresh && + bd->bd_freebuffers > bd->bd_lofreebuffers) { + msleep(&bd->bd_running, BD_RUN_LOCKPTR(bd), + PRIBIO, "-", hz); + } else { + /* Avoid spurious wakeups while running. */ + atomic_store_int(&bd->bd_running, 1); + } } + wakeup(&bd->bd_shutdown); + BD_RUN_UNLOCK(bd); + kthread_exit(); } /* @@ -3382,6 +3398,21 @@ buf_flush(struct vnode *vp, struct bufdomain *bd, int target) return (flushed); } +static void +buf_daemon_shutdown(void *arg __unused, int howto __unused) +{ + int error; + + mtx_lock(&bdlock); + bd_shutdown = true; + wakeup(&bd_request); + error = msleep(&bd_shutdown, &bdlock, 0, "buf_daemon_shutdown", + 60 * hz); + mtx_unlock(&bdlock); + if (error != 0) + printf("bufdaemon wait error: %d\n", error); +} + static void buf_daemon() { @@ -3393,7 +3424,7 @@ buf_daemon() /* * This process needs to be suspended prior to shutdown sync. */ - EVENTHANDLER_REGISTER(shutdown_pre_sync, kthread_shutdown, curthread, + EVENTHANDLER_REGISTER(shutdown_pre_sync, buf_daemon_shutdown, NULL, SHUTDOWN_PRI_LAST + 100); /* @@ -3413,12 +3444,10 @@ buf_daemon() */ curthread->td_pflags |= TDP_NORUNNINGBUF | TDP_BUFNEED; mtx_lock(&bdlock); - for (;;) { + while (!bd_shutdown) { bd_request = 0; mtx_unlock(&bdlock); - kthread_suspend_check(); - /* * Save speedupreq for this pass and reset to capture new * requests. @@ -3455,6 +3484,8 @@ buf_daemon() * to avoid endless loops on unlockable buffers. */ mtx_lock(&bdlock); + if (bd_shutdown) + break; if (BIT_EMPTY(BUF_DOMAINS, &bdlodirty)) { /* * We reached our low water mark, reset the @@ -3480,6 +3511,9 @@ buf_daemon() msleep(&bd_request, &bdlock, PVM, "qsleep", hz / 10); } } + wakeup(&bd_shutdown); + mtx_unlock(&bdlock); + kthread_exit(); } /*