From nobody Tue May 16 21:38:48 2023 X-Original-To: dev-commits-src-branches@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4QLV3s0fHVz4BTpg; Tue, 16 May 2023 21:38:49 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4QLV3r6zM7z4PGn; Tue, 16 May 2023 21:38:48 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1684273129; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=ct0uZ/O+Pf2ZIyuwzk/wlp2bX6xOlW4n/Yi69zliDNM=; b=WF+WJFYFHGI2n3HUTBUKBZNfo8a8AWv9snQqmB7Jeq3RYvGsQtT4xxL97xi72lJSG0omJT 0w3K9P3sZqzaRrX0aAKxOtNo8P+r/sDNtnDxWpyTTcAyuE3yxKcw0RAdxNjN2ScFTVO5x+ k9qRVSgR9rjjfKOMyKI9/XcDuhVFc6p0kfQshpRaNti2KHjAPmdpFMjKpg3uqee2HleLCf vyi4U31EegbhbNOf4wXOq1Mz+O9OoGE/xClcciesGI1pJPAvBmJoifz9HNFyCRafWCRShO QZS3t5Tbj4SFZEEdnZ/tMgufD3wwJLKzLjzT2LU8PMnFm5cVkZtTocTLnofgGA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1684273129; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=ct0uZ/O+Pf2ZIyuwzk/wlp2bX6xOlW4n/Yi69zliDNM=; b=nPyLO2YXZHaSwE2AyA1fAWoyLeRzK0aZD6shRpBQbgf838X9Tuqnxw475WOrrTIYtQzrRG Gt66tALgUfYyLp4AVI1nah5HnrP05NuHtw4yeUcD9rIq24B8FcqU87nbzPCRO3AmGnQMga iz176euKr66pevW3P+YXXbnJYGYZllpwN3keVrDJFAkm0wgjF7LWiCCqbHIXwk87mAerrN Anh560ZlVB7n0mgk0gV3KZqGnnhogGeZRcUq1zXgoYKDp6VwE6KMuC2/mt7L0Heu5EfjRg rN+L6DNOSgYxFjhfqfV7xlezZkgdHnB0LZLfEljFRB4qgELIvFpTGNlMaS2o0g== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1684273129; a=rsa-sha256; cv=none; b=uf+Ajc+DhLPyiFDyQQvmWF6JdNMQ8z5A9py2Gzvw4wFn2CkiaL9TKqQzW2FMXivS3xb7IV NXFGSljsKjKHmVegCvrU+M/A9lGs9G15QYtVrE59UTQUF4t/fVQIDgYNRV6e0qsjFuRcuO Zdw5BbG1lMVV8KXQtkFKA/KkAF1c6jfd3XF3Pv0WFqcCr8tuaWUkLyj7w4qbhGncLSNaWc K3KIsvlvFo0eC4H/ZusdKjH0KC14uiYPRIoRofjzQ13zrcs8on8PPGG/+sE2UlIEDnN3ab fLd0AczzaBoe9XnK529h7d9eOSMAX5/tRdC5hGqi27D+FMjaWDY7nfExAFbe3w== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4QLV3r5xXszdWP; Tue, 16 May 2023 21:38:48 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 34GLcmJ0021063; Tue, 16 May 2023 21:38:48 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 34GLcmM1021062; Tue, 16 May 2023 21:38:48 GMT (envelope-from git) Date: Tue, 16 May 2023 21:38:48 GMT Message-Id: <202305162138.34GLcmM1021062@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Rick Macklem Subject: git: 4ee3d271b74d - stable/13 - nfsd: Add NFSD_VNET() macros to nfs_fha_new.c List-Id: Commits to the stable branches of the FreeBSD src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-branches List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-branches@freebsd.org X-BeenThere: dev-commits-src-branches@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: rmacklem X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 4ee3d271b74dbaf710adfd11432c12d3b21f2628 Auto-Submitted: auto-generated X-ThisMailContainsUnwantedMimeParts: N The branch stable/13 has been updated by rmacklem: URL: https://cgit.FreeBSD.org/src/commit/?id=4ee3d271b74dbaf710adfd11432c12d3b21f2628 commit 4ee3d271b74dbaf710adfd11432c12d3b21f2628 Author: Rick Macklem AuthorDate: 2023-02-18 15:45:38 +0000 Commit: Rick Macklem CommitDate: 2023-05-16 21:37:39 +0000 nfsd: Add NFSD_VNET() macros to nfs_fha_new.c Although it is not 100% obvious if the affinity threads lists need to be vnet'd when nfsd runs in a prison, I think it is necessary for the unusual case where the same file system is exported in multiple prisons. For this case, the affinity code might try to assign the RPC to a svc thread that runs in a different prison. Also, it makes sense to vnet them, since there are separate svc threads for each prison running nfsd(8). This patch adds the macros for vnet'ng to nfs_fha_new.c. The macros are still null, so the only semantics change is malloc'ng the fha_param structure. (cherry picked from commit cd406ac94d8beae7f184adb14a3c94e058366b9a) --- sys/fs/nfsserver/nfs_fha_new.c | 108 +++++++++++++++++++++++------------------ 1 file changed, 62 insertions(+), 46 deletions(-) diff --git a/sys/fs/nfsserver/nfs_fha_new.c b/sys/fs/nfsserver/nfs_fha_new.c index 05a03bee5d1c..203e98d3ac86 100644 --- a/sys/fs/nfsserver/nfs_fha_new.c +++ b/sys/fs/nfsserver/nfs_fha_new.c @@ -56,14 +56,14 @@ static int fhenew_stats_sysctl(SYSCTL_HANDLER_ARGS); static void fha_extract_info(struct svc_req *req, struct fha_info *i); -static struct fha_params fhanew_softc; +NFSD_VNET_DEFINE_STATIC(struct fha_params *, fhanew_softc); SYSCTL_DECL(_vfs_nfsd); extern int newnfs_nfsv3_procid[]; -SYSINIT(nfs_fhanew, SI_SUB_ROOT_CONF, SI_ORDER_ANY, fhanew_init, NULL); -SYSUNINIT(nfs_fhanew, SI_SUB_ROOT_CONF, SI_ORDER_ANY, fhanew_uninit, NULL); +SYSINIT(nfs_fhanew, SI_SUB_VNET_DONE, SI_ORDER_ANY, fhanew_init, NULL); +SYSUNINIT(nfs_fhanew, SI_SUB_VNET_DONE, SI_ORDER_ANY, fhanew_uninit, NULL); static void fhanew_init(void *foo) @@ -71,9 +71,9 @@ fhanew_init(void *foo) struct fha_params *softc; int i; - softc = &fhanew_softc; - - bzero(softc, sizeof(*softc)); + NFSD_VNET(fhanew_softc) = malloc(sizeof(struct fha_params), M_TEMP, + M_WAITOK | M_ZERO); + softc = NFSD_VNET(fhanew_softc); snprintf(softc->server_name, sizeof(softc->server_name), FHANEW_SERVER_NAME); @@ -82,12 +82,14 @@ fhanew_init(void *foo) * Initialize the sysctl context list for the fha module. */ sysctl_ctx_init(&softc->sysctl_ctx); - softc->sysctl_tree = SYSCTL_ADD_NODE(&softc->sysctl_ctx, - SYSCTL_STATIC_CHILDREN(_vfs_nfsd), OID_AUTO, "fha", - CTLFLAG_RD | CTLFLAG_MPSAFE, 0, "NFS File Handle Affinity (FHA)"); - if (softc->sysctl_tree == NULL) { - printf("%s: unable to allocate sysctl tree\n", __func__); - return; + if (IS_DEFAULT_VNET(curvnet)) { + softc->sysctl_tree = SYSCTL_ADD_NODE(&softc->sysctl_ctx, + SYSCTL_STATIC_CHILDREN(_vfs_nfsd), OID_AUTO, "fha", + CTLFLAG_RD | CTLFLAG_MPSAFE, 0, "NFS File Handle Affinity (FHA)"); + if (softc->sysctl_tree == NULL) { + printf("%s: unable to allocate sysctl tree\n", __func__); + return; + } } for (i = 0; i < FHA_HASH_SIZE; i++) @@ -106,36 +108,38 @@ fhanew_init(void *foo) /* * Add sysctls so the user can change the tuning parameters. */ - SYSCTL_ADD_UINT(&softc->sysctl_ctx, SYSCTL_CHILDREN(softc->sysctl_tree), - OID_AUTO, "enable", CTLFLAG_RWTUN, - &softc->ctls.enable, 0, "Enable NFS File Handle Affinity (FHA)"); - - SYSCTL_ADD_UINT(&softc->sysctl_ctx, SYSCTL_CHILDREN(softc->sysctl_tree), - OID_AUTO, "read", CTLFLAG_RWTUN, - &softc->ctls.read, 0, "Enable NFS FHA read locality"); - - SYSCTL_ADD_UINT(&softc->sysctl_ctx, SYSCTL_CHILDREN(softc->sysctl_tree), - OID_AUTO, "write", CTLFLAG_RWTUN, - &softc->ctls.write, 0, "Enable NFS FHA write locality"); - - SYSCTL_ADD_UINT(&softc->sysctl_ctx, SYSCTL_CHILDREN(softc->sysctl_tree), - OID_AUTO, "bin_shift", CTLFLAG_RWTUN, - &softc->ctls.bin_shift, 0, - "Maximum locality distance 2^(bin_shift) bytes"); - - SYSCTL_ADD_UINT(&softc->sysctl_ctx, SYSCTL_CHILDREN(softc->sysctl_tree), - OID_AUTO, "max_nfsds_per_fh", CTLFLAG_RWTUN, - &softc->ctls.max_nfsds_per_fh, 0, "Maximum nfsd threads that " - "should be working on requests for the same file handle"); - - SYSCTL_ADD_UINT(&softc->sysctl_ctx, SYSCTL_CHILDREN(softc->sysctl_tree), - OID_AUTO, "max_reqs_per_nfsd", CTLFLAG_RWTUN, - &softc->ctls.max_reqs_per_nfsd, 0, "Maximum requests that " - "single nfsd thread should be working on at any time"); - - SYSCTL_ADD_OID(&softc->sysctl_ctx, SYSCTL_CHILDREN(softc->sysctl_tree), - OID_AUTO, "fhe_stats", CTLTYPE_STRING | CTLFLAG_RD | CTLFLAG_MPSAFE, - 0, 0, fhenew_stats_sysctl, "A", ""); + if (IS_DEFAULT_VNET(curvnet)) { + SYSCTL_ADD_UINT(&softc->sysctl_ctx, SYSCTL_CHILDREN(softc->sysctl_tree), + OID_AUTO, "enable", CTLFLAG_RWTUN, + &softc->ctls.enable, 0, "Enable NFS File Handle Affinity (FHA)"); + + SYSCTL_ADD_UINT(&softc->sysctl_ctx, SYSCTL_CHILDREN(softc->sysctl_tree), + OID_AUTO, "read", CTLFLAG_RWTUN, + &softc->ctls.read, 0, "Enable NFS FHA read locality"); + + SYSCTL_ADD_UINT(&softc->sysctl_ctx, SYSCTL_CHILDREN(softc->sysctl_tree), + OID_AUTO, "write", CTLFLAG_RWTUN, + &softc->ctls.write, 0, "Enable NFS FHA write locality"); + + SYSCTL_ADD_UINT(&softc->sysctl_ctx, SYSCTL_CHILDREN(softc->sysctl_tree), + OID_AUTO, "bin_shift", CTLFLAG_RWTUN, + &softc->ctls.bin_shift, 0, + "Maximum locality distance 2^(bin_shift) bytes"); + + SYSCTL_ADD_UINT(&softc->sysctl_ctx, SYSCTL_CHILDREN(softc->sysctl_tree), + OID_AUTO, "max_nfsds_per_fh", CTLFLAG_RWTUN, + &softc->ctls.max_nfsds_per_fh, 0, "Maximum nfsd threads that " + "should be working on requests for the same file handle"); + + SYSCTL_ADD_UINT(&softc->sysctl_ctx, SYSCTL_CHILDREN(softc->sysctl_tree), + OID_AUTO, "max_reqs_per_nfsd", CTLFLAG_RWTUN, + &softc->ctls.max_reqs_per_nfsd, 0, "Maximum requests that " + "single nfsd thread should be working on at any time"); + + SYSCTL_ADD_OID(&softc->sysctl_ctx, SYSCTL_CHILDREN(softc->sysctl_tree), + OID_AUTO, "fhe_stats", CTLTYPE_STRING | CTLFLAG_RD | CTLFLAG_MPSAFE, + 0, 0, fhenew_stats_sysctl, "A", ""); + } } static void @@ -144,11 +148,12 @@ fhanew_uninit(void *foo) struct fha_params *softc; int i; - softc = &fhanew_softc; + softc = NFSD_VNET(fhanew_softc); sysctl_ctx_free(&softc->sysctl_ctx); for (i = 0; i < FHA_HASH_SIZE; i++) mtx_destroy(&softc->fha_hash[i].mtx); + free(softc, M_TEMP); } static rpcproc_t @@ -556,11 +561,13 @@ noloc: SVCTHREAD * fhanew_assign(SVCTHREAD *this_thread, struct svc_req *req) { - struct fha_params *softc = &fhanew_softc; + struct fha_params *softc; SVCTHREAD *thread; struct fha_info i; struct fha_hash_entry *fhe; + NFSD_CURVNET_SET(NFSD_TD_TO_VNET(curthread)); + softc = NFSD_VNET(fhanew_softc); /* Check to see whether we're enabled. */ if (softc->ctls.enable == 0) goto thist; @@ -602,9 +609,11 @@ fhanew_assign(SVCTHREAD *this_thread, struct svc_req *req) mtx_lock(&thread->st_lock); mtx_unlock(fhe->mtx); + NFSD_CURVNET_RESTORE(); return (thread); thist: req->rq_p1 = NULL; + NFSD_CURVNET_RESTORE(); mtx_lock(&this_thread->st_lock); return (this_thread); } @@ -619,12 +628,15 @@ fhanew_nd_complete(SVCTHREAD *thread, struct svc_req *req) struct fha_hash_entry *fhe = req->rq_p1; struct mtx *mtx; + NFSD_CURVNET_SET(NFSD_TD_TO_VNET(curthread)); /* * This may be called for reqs that didn't go through * fha_assign (e.g. extra NULL ops used for RPCSEC_GSS. */ - if (!fhe) + if (!fhe) { + NFSD_CURVNET_RESTORE(); return; + } mtx = fhe->mtx; mtx_lock(mtx); @@ -638,12 +650,13 @@ fhanew_nd_complete(SVCTHREAD *thread, struct svc_req *req) fha_hash_entry_remove(fhe); } mtx_unlock(mtx); + NFSD_CURVNET_RESTORE(); } static int fhenew_stats_sysctl(SYSCTL_HANDLER_ARGS) { - struct fha_params *softc = &fhanew_softc; + struct fha_params *softc; int error, i; struct sbuf sb; struct fha_hash_entry *fhe; @@ -652,6 +665,8 @@ fhenew_stats_sysctl(SYSCTL_HANDLER_ARGS) sbuf_new(&sb, NULL, 65536, SBUF_FIXEDLEN); + NFSD_CURVNET_SET(NFSD_TD_TO_VNET(curthread)); + softc = NFSD_VNET(fhanew_softc); for (i = 0; i < FHA_HASH_SIZE; i++) if (!LIST_EMPTY(&softc->fha_hash[i].list)) break; @@ -694,6 +709,7 @@ fhenew_stats_sysctl(SYSCTL_HANDLER_ARGS) } out: + NFSD_CURVNET_RESTORE(); sbuf_trim(&sb); sbuf_finish(&sb); error = sysctl_handle_string(oidp, sbuf_data(&sb), sbuf_len(&sb), req);