From nobody Wed May 21 10:08:28 2025 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4b2Rvd1JZtz5x8Zs; Wed, 21 May 2025 10:08:29 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4b2Rvd0CJwz3DTh; Wed, 21 May 2025 10:08:29 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1747822109; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=/u0GjZEWTs65rgvTAhatbvusOncvaMjh0ae7fpon0qc=; b=H58g1MYv48L3z9q87l0LRPac7UJQTRXxdnY/xpsQDNW6bxyY4Y0JLmK/oEuLCi2u5f+dNn cBsnJLit7PaNS2VI3EwF1WBWs6upc0E46bQP95NCSi3ApK3JtjWL32bZDw17PW+E/G6gIO OutkGcr0R4fqAbAWsDB9bJdokCrFBogwx6qhkXzRnNIjRGeHgUk8Vm3kTMN/EAO1Jv2/zg VksgvGTkGFLQwb/VG/N2nnnGRLmlmuKz3jgMfhkfQWRopWeJdmnj9/hqVGdybugBcR6oiu Q3iqD+lcoaR70Pyab5MaRVNXP8Yahg9tQtfHaoQbbQjcCgxwixq9FOUovyn05A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1747822109; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=/u0GjZEWTs65rgvTAhatbvusOncvaMjh0ae7fpon0qc=; b=M0X3NOhvwrSnCG55j37bQLA9t3fFo/wLCQOkzTh1MgjI2laNvQv9VLtKFjXv+kpBxGtjOa zC3gmiqvuqBOP7Ki4jb8alWg9lKRCR5w+69gbVJdBVAl6fzNfNZmBVB/H8qSMzZgNvyh2m g4HPMWsnaFmcMoFVR+7rQJFvU+x5BYxBzkf+2+VxptmP+9IqAVB3vhAnAchwUCVo467969 5pE/gGZMLLZl3DSukBrFKEWqImbOghhgKVl8Ovwp5CQ2199JXHRckzVrsm0wZUNRkTNMpk lGBCVPAJ4ef8RztQdO684dxgpD1yHfcbpFITHiC9TLz2RPsdz0pyT4gizXn9+w== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1747822109; a=rsa-sha256; cv=none; b=SgS9vcqfk/eN2LIgh5GM5LEA4nVr7DSid0P5uim7AngQpFDSQFVH4ssaoiB2f7QYA1wPP6 zjaO9xDQsRZw/sq9deyQ+8Y++f6KEQLjDHJgzo/iTnT5SHBzY8ESKD/XeffFOiO3vXLSf5 +cD6A4NtIN3HzSxC4NRRJYy8+R3drWiQmSh86Mn7i37D6lPnUxwnQv960v1F9nXFwS90mQ z6xjHPyzQJcDcJBQzp/bXulw4PkFQfOa9E1XQ1+JxSCU9qBxik3HJ2Da/9Pa9vAthUrVX7 25D1fwsshH59qloHBV5wsF+2TDuX6bb3Wh1RV9kqafTRj99LCWHnu3qFSY45qg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4b2Rvc6wH8zVS7; Wed, 21 May 2025 10:08:28 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 54LA8S9f011762; Wed, 21 May 2025 10:08:28 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 54LA8SYx011758; Wed, 21 May 2025 10:08:28 GMT (envelope-from git) Date: Wed, 21 May 2025 10:08:28 GMT Message-Id: <202505211008.54LA8SYx011758@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Olivier Certner Subject: git: 7a9ea03e4bbf - releng/14.3 - vfs: vn_alloc(): Stop always calling vn_alloc_hard() and direct reclaiming List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: olce X-Git-Repository: src X-Git-Refname: refs/heads/releng/14.3 X-Git-Reftype: branch X-Git-Commit: 7a9ea03e4bbfee1b2192d9a5b4da89a53d3a2c14 Auto-Submitted: auto-generated The branch releng/14.3 has been updated by olce: URL: https://cgit.FreeBSD.org/src/commit/?id=7a9ea03e4bbfee1b2192d9a5b4da89a53d3a2c14 commit 7a9ea03e4bbfee1b2192d9a5b4da89a53d3a2c14 Author: Olivier Certner AuthorDate: 2025-05-13 13:47:42 +0000 Commit: Olivier Certner CommitDate: 2025-05-21 10:07:15 +0000 vfs: vn_alloc(): Stop always calling vn_alloc_hard() and direct reclaiming Commit ab05a1cf321aca0f intended to revert commit 8733bc277a383cf5 ("vfs: don't provoke recycling non-free vnodes without a good reason"), but due to intervening changes in commit 054f45e026d898bd ("vfs: further speed up continuous free vnode recycle"), it also had to revert part of it. In particular, while removing the whole 'if (vn_alloc_cyclecount != 0)' block, it inadvertantly removed the code block resetting 'vn_alloc_cyclecount' to 0 and skipping direct vnode reclamation (done further below through vnlru_free_locked_direct()), which had been outside the 'if' before the intervening commit. Removing this block instead of reinstating it in practice causes 'vn_alloc_cyclecount' to (almost) never be 0, making vn_alloc() always call vn_alloc_hard(), which takes the 'vnode_list_mtx' mutex. In other words, this disables the fast path. [The reverted commit, which introduced the 'if (vn_alloc_cyclecount != 0)' guarding this block, actually never executed it because it also had the bug that 'vn_alloc_cyclecount' would always stay at 0, hiding its usefulness.] Additionally, not skipping direct vnode reclamation even when there are less vnodes than 'kern.maxvnodes' not only causes unnecessary contention but also plain livelocks as vnlru_free_locked_direct() does not itself check whether there are actually "free" (not referenced) vnodes to be deallocated, and will blindly browse all the vnode list until it finds one (which it may not, or only a few ones at the end). As the fast path was disabled, all threads in the system would soon be competing for the vnode list lock, outpacing the vnlru process that could never actually recycle vnodes in a more agressive manner (i.e., even if they have a non-zero hold count). And we could more easily get into this situation, as each vnode allocation was reducing the count of "free" vnodes, even if entirely new vnodes could be allocated instead. This part was mitigated by the vnlru process (before the tipping point described above), which explains why the mechanism would not always livelock. Not skipping direct vnode reclamation was arguably a bug introduced by the intervening commit (054f45e026d898bd), but was mitigated by vn_alloc_hard() not being called in the fast path. The revert commit, by disabling the fast path, made it significantly annoying (to the point of getting a few livelocks a week in some of my workloads). Restore the reset of 'vn_alloc_cyclecount' to 0 and skip direct reclamation when the current number of vnodes is below the 'kern.maxvnodes' limit, indicating we can start allocating a new 'struct vnode' right away. While here, fix the comparison with the limit when 'bumped' is true. Reviewed by: kib (older version), avg Tested by: avg, olce Fixes: ab05a1cf321aca0f (revert of "vfs: don't provoke recycling non-free vnodes without a good reason") Fixes: 054f45e026d898bd ("vfs: further speed up continuous free vnode recycle") MFC after: 5 days Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D50338 (cherry picked from commit d90c9c24e2c3701949c47061b5ad198eedeebfb9) (cherry picked from commit ce6b7a5919da518715d6163baa42035d6d2c1bd0) Approved by: re (cperciva) --- sys/kern/vfs_subr.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/sys/kern/vfs_subr.c b/sys/kern/vfs_subr.c index 009d305b8c1f..a640bede85b9 100644 --- a/sys/kern/vfs_subr.c +++ b/sys/kern/vfs_subr.c @@ -1964,11 +1964,24 @@ vn_alloc_hard(struct mount *mp, u_long rnumvnodes, bool bumped) mtx_lock(&vnode_list_mtx); + /* + * Reload 'numvnodes', as since we acquired the lock, it may have + * changed significantly if we waited, and 'rnumvnodes' above was only + * actually passed if 'bumped' is true (else it is 0). + */ + rnumvnodes = atomic_load_long(&numvnodes); + if (rnumvnodes + !bumped < desiredvnodes) { + vn_alloc_cyclecount = 0; + mtx_unlock(&vnode_list_mtx); + goto alloc; + } + rfreevnodes = vnlru_read_freevnodes(); if (vn_alloc_cyclecount++ >= rfreevnodes) { vn_alloc_cyclecount = 0; vstir = true; } + /* * Grow the vnode cache if it will not be above its target max * after growing. Otherwise, if the free list is nonempty, try