From nobody Mon Nov 01 02:08:33 2021 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 52D651827363; Mon, 1 Nov 2021 02:08:34 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4HjGfV1hwLz3KLM; Mon, 1 Nov 2021 02:08:34 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 1A3E9E2C; Mon, 1 Nov 2021 02:08:34 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 1A128YEi012418; Mon, 1 Nov 2021 02:08:34 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 1A128XRp012417; Mon, 1 Nov 2021 02:08:33 GMT (envelope-from git) Date: Mon, 1 Nov 2021 02:08:33 GMT Message-Id: <202111010208.1A128XRp012417@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Rick Macklem Subject: git: 9910716d014a - stable/13 - nfscl: Handle NFSv4.1/4.2 Close RPC NFSERR_DELAY replies better List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: rmacklem X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 9910716d014acc8477f1508bc0a1395d823c1a6f Auto-Submitted: auto-generated X-ThisMailContainsUnwantedMimeParts: N The branch stable/13 has been updated by rmacklem: URL: https://cgit.FreeBSD.org/src/commit/?id=9910716d014acc8477f1508bc0a1395d823c1a6f commit 9910716d014acc8477f1508bc0a1395d823c1a6f Author: Rick Macklem AuthorDate: 2021-10-18 22:02:21 +0000 Commit: Rick Macklem CommitDate: 2021-11-01 02:02:53 +0000 nfscl: Handle NFSv4.1/4.2 Close RPC NFSERR_DELAY replies better Without this patch, if a NFSv4.1/4.2 server replies NFSERR_DELAY to a Close operation, the client loops retrying the Close while holding a shared lock on the clientID. This shared lock blocks returns of delegations, even though the server has issued a CB_RECALL to request the delegation return. This patch delays doing a retry of a Close that received a reply of NFSERR_DELAY until after the shared lock on the clientID is released, for NFSv4.1/4.2. To fix this for NFSv4.0 would be very difficult and since the only known NFSv4 server to reply NFSERR_DELAY to Close only does NFSv4.1/4.2, this fix is hoped to be sufficient. This problem was detected during a recent IETF working group NFSv4 testing event. (cherry picked from commit 52dee2bc035545f7ae2b838d8a0449f65043cd8a) --- sys/fs/nfs/nfs_var.h | 3 ++- sys/fs/nfsclient/nfs_clrpcops.c | 11 +++++++---- sys/fs/nfsclient/nfs_clstate.c | 23 +++++++++++++++++++++-- 3 files changed, 30 insertions(+), 7 deletions(-) diff --git a/sys/fs/nfs/nfs_var.h b/sys/fs/nfs/nfs_var.h index 609debe9cc16..44aedec09a96 100644 --- a/sys/fs/nfs/nfs_var.h +++ b/sys/fs/nfs/nfs_var.h @@ -600,7 +600,8 @@ void nfscl_dumpstate(struct nfsmount *, int, int, int, int); void nfscl_dupopen(vnode_t, int); int nfscl_getclose(vnode_t, struct nfsclclient **); int nfscl_doclose(vnode_t, struct nfsclclient **, NFSPROC_T *); -void nfsrpc_doclose(struct nfsmount *, struct nfsclopen *, NFSPROC_T *); +int nfsrpc_doclose(struct nfsmount *, struct nfsclopen *, NFSPROC_T *, bool, + bool); int nfscl_deleg(mount_t, struct nfsclclient *, u_int8_t *, int, struct ucred *, NFSPROC_T *, struct nfscldeleg **); void nfscl_lockinit(struct nfsv4lock *); diff --git a/sys/fs/nfsclient/nfs_clrpcops.c b/sys/fs/nfsclient/nfs_clrpcops.c index ab25306a9c29..211fd3523f34 100644 --- a/sys/fs/nfsclient/nfs_clrpcops.c +++ b/sys/fs/nfsclient/nfs_clrpcops.c @@ -758,8 +758,9 @@ nfsrpc_close(vnode_t vp, int doclose, NFSPROC_T *p) /* * Close the open. */ -void -nfsrpc_doclose(struct nfsmount *nmp, struct nfsclopen *op, NFSPROC_T *p) +int +nfsrpc_doclose(struct nfsmount *nmp, struct nfsclopen *op, NFSPROC_T *p, + bool loop_on_delayed, bool freeop) { struct nfsrv_descript nfsd, *nd = &nfsd; struct nfscllockowner *lp, *nlp; @@ -838,7 +839,7 @@ nfsrpc_doclose(struct nfsmount *nmp, struct nfsclopen *op, NFSPROC_T *p) nfscl_lockexcl(&op->nfso_own->nfsow_rwlock, NFSCLSTATEMUTEXPTR); NFSUNLOCKCLSTATE(); do { - error = nfscl_tryclose(op, tcred, nmp, p, true); + error = nfscl_tryclose(op, tcred, nmp, p, loop_on_delayed); if (error == NFSERR_GRACE) (void) nfs_catnap(PZERO, error, "nfs_close"); } while (error == NFSERR_GRACE); @@ -847,9 +848,11 @@ nfsrpc_doclose(struct nfsmount *nmp, struct nfsclopen *op, NFSPROC_T *p) LIST_FOREACH_SAFE(lp, &op->nfso_lock, nfsl_list, nlp) nfscl_freelockowner(lp, 0); - nfscl_freeopen(op, 0, true); + if (freeop && error != NFSERR_DELAY) + nfscl_freeopen(op, 0, true); NFSUNLOCKCLSTATE(); NFSFREECRED(tcred); + return (error); } /* diff --git a/sys/fs/nfsclient/nfs_clstate.c b/sys/fs/nfsclient/nfs_clstate.c index 984d8382ee45..3b76cbe502c4 100644 --- a/sys/fs/nfsclient/nfs_clstate.c +++ b/sys/fs/nfsclient/nfs_clstate.c @@ -3329,8 +3329,10 @@ int nfscl_doclose(vnode_t vp, struct nfsclclient **clpp, NFSPROC_T *p) { struct nfsclclient *clp; + struct nfsmount *nmp; struct nfsclowner *owp, *nowp; - struct nfsclopen *op; + struct nfsclopen *op, *nop; + struct nfsclopenhead delayed; struct nfscldeleg *dp; struct nfsfh *nfhp; struct nfsclrecalllayout *recallp; @@ -3341,6 +3343,7 @@ nfscl_doclose(vnode_t vp, struct nfsclclient **clpp, NFSPROC_T *p) return (error); *clpp = clp; + nmp = VFSTONFS(vp->v_mount); nfhp = VTONFS(vp)->n_fhp; recallp = malloc(sizeof(*recallp), M_NFSLAYRECALL, M_WAITOK); NFSLOCKCLSTATE(); @@ -3365,6 +3368,7 @@ nfscl_doclose(vnode_t vp, struct nfsclclient **clpp, NFSPROC_T *p) nfscl_retoncloselayout(vp, clp, nfhp->nfh_fh, nfhp->nfh_len, &recallp); /* Now process the opens against the server. */ + LIST_INIT(&delayed); lookformore: LIST_FOREACH(op, NFSCLOPENHASH(clp, nfhp->nfh_fh, nfhp->nfh_len), nfso_hash) { @@ -3378,8 +3382,16 @@ lookformore: op->nfso_opencnt)); #endif NFSUNLOCKCLSTATE(); - nfsrpc_doclose(VFSTONFS(vp->v_mount), op, p); + if (NFSHASNFSV4N(nmp)) + error = nfsrpc_doclose(nmp, op, p, false, true); + else + error = nfsrpc_doclose(nmp, op, p, true, true); NFSLOCKCLSTATE(); + if (error == NFSERR_DELAY) { + nfscl_unlinkopen(op); + op->nfso_own = NULL; + LIST_INSERT_HEAD(&delayed, op, nfso_list); + } goto lookformore; } } @@ -3390,6 +3402,13 @@ lookformore: * used by the function, but calling free() with a NULL pointer is ok. */ free(recallp, M_NFSLAYRECALL); + + /* Now, loop retrying the delayed closes. */ + LIST_FOREACH_SAFE(op, &delayed, nfso_list, nop) { + nfsrpc_doclose(nmp, op, p, true, false); + LIST_REMOVE(op, nfso_list); + nfscl_freeopen(op, 0, false); + } return (0); }