From nobody Thu Jun 09 22:45:50 2022 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 69C8C83799B; Thu, 9 Jun 2022 22:45:50 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4LJzhZ2Pl4z4YXy; Thu, 9 Jun 2022 22:45:50 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1654814750; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=CM+BfxSQ8trMaeWvZEBc1lgUzxCvHMK5bsoep60OWWE=; b=LnRMwwvHyEvFvhQz6aaAdzUXTfcc+3I+3l+oUR4JxAVt6UqNrzIgjXldACwf4DwkHAQzNF gaKZdTBV5jfuCpwgQY9mxpUifSo8q1WkL0aNnabe5xKwoHWphTRU95NWU01bDGBHQksPle ebQKs5c3RZoeUyf5s9fOGSxEVsK8e36ju75fWxM1zpAA8IoTbpINgoh/aFbylh4bH8BfML JOAapL2G5NgXUeqzC2lGA5aM37ktbqnlAlFdTbMHqvm9LLqaf6acQ5YkBJCOrDRwBQl2iL IaAZgXfPxLTmoT06xhtBIXyOQUlUVG7KhUcTrN2Qxk8vxlvb/ttc1/qr6LPoEg== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 2EA3947EF; Thu, 9 Jun 2022 22:45:50 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 259Mjork000131; Thu, 9 Jun 2022 22:45:50 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 259MjocU000130; Thu, 9 Jun 2022 22:45:50 GMT (envelope-from git) Date: Thu, 9 Jun 2022 22:45:50 GMT Message-Id: <202206092245.259MjocU000130@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Rick Macklem Subject: git: 8929caf5587b - stable/13 - nfscl: Add support for a NFSv4 AppendWrite RPC List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: rmacklem X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 8929caf5587b26798caba6435d500a26158afbda Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1654814750; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=CM+BfxSQ8trMaeWvZEBc1lgUzxCvHMK5bsoep60OWWE=; b=rdoQv51Qtp1lmLTurfWkiKNK2HimIpxkwrLKnABFZ2blWNYvlTmVQOXKhT6NIJZtg9rw7u 3MFrKYfaF+O70UtRcPX4ZAPpqtC0ScPDQoangB/l/FvP/UR193tpu1TnMBsHO1aXZapw4a 1Nde2aUx9hA2hLlTFVGQL0r/XeI/itlITr/GY4DD5SFIl5D6KVTQ+Zx8InrKOt8Q9sA0Hg 9mjvd8JSK8FfiUXiHUkjs+dkLYna5z5FSzekzb8+wEKIGmJJzEcLivI01UpGdl5HXobBvV OO3Qp2sMStAPg8Hit4edCJ6rcDgpGNEgbC2tJZ0EbRzskhZF1VZGjMb7CUKyag== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1654814750; a=rsa-sha256; cv=none; b=Xwdbavi54x/JW097khJgDuEfczcXXPVuhirprCe/9UzZAY34MvRq9vySHEKQFkNbZfWb8k YXt7YQ79M/9LlVMwbKShbmcwEsCL7Vp9PXpEUmzguTSzekAzJJ13c4dSNLAeG8JhsWRT/R XZPaBQw5kyvYgM9qq//G3/XUT0/J6X8h5ruQTNDP2QZMz5IsV6+qkyfJHkSVG1EfhdJz1T S2KuIVaJxNS9tyQRm8F5oAKYQ4pc/6Ly8Zg0f5LT53t4BQK/UBpH8eu198pyuXNiQR4HID K2AElVKAdhcX2q4HMuuynhfgCWxN/xRxqM/BeLLCaL9TWeSZGzWWMVKrqbHyvw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch stable/13 has been updated by rmacklem: URL: https://cgit.FreeBSD.org/src/commit/?id=8929caf5587b26798caba6435d500a26158afbda commit 8929caf5587b26798caba6435d500a26158afbda Author: Rick Macklem AuthorDate: 2022-04-30 20:49:23 +0000 Commit: Rick Macklem CommitDate: 2022-06-09 20:46:14 +0000 nfscl: Add support for a NFSv4 AppendWrite RPC For IO_APPEND VOP_WRITE()s, the code first does a Getattr RPC to acquire the file's size, before it can do the Write RPC. Although NFS does not have an append write operation, an NFSv4 compound can use a Verify operation to check that the client's notion of the file's size is correct, followed by the Write operation. This patch modifies the NFSv4 client to use an Appendwrite RPC, which does a Verify to check the file's size before doing the Write. This avoids the need for a Getattr RPC to preceed this RPC and reduces the RPC count by half for IO_APPEND writes, so long as the client knows the file's size. The nfsd structure was moved from the stack to be malloc()'d, since the kernel stack limit was being exceeded. While here, fix the types of a few variables, although there should not be any semantics change caused by these type changes. (cherry picked from commit 5218d82c81f987223054671b9821d39f08d87599) --- sys/fs/nfs/nfs_var.h | 2 +- sys/fs/nfsclient/nfs.h | 2 +- sys/fs/nfsclient/nfs_clbio.c | 26 +++++++++++----- sys/fs/nfsclient/nfs_clrpcops.c | 69 +++++++++++++++++++++++++++++++++-------- sys/fs/nfsclient/nfs_clvnops.c | 6 ++-- 5 files changed, 79 insertions(+), 26 deletions(-) diff --git a/sys/fs/nfs/nfs_var.h b/sys/fs/nfs/nfs_var.h index 7bdcbcd50575..69bba223b4b5 100644 --- a/sys/fs/nfs/nfs_var.h +++ b/sys/fs/nfs/nfs_var.h @@ -470,7 +470,7 @@ int nfsrpc_readlink(vnode_t, struct uio *, struct ucred *, int nfsrpc_read(vnode_t, struct uio *, struct ucred *, NFSPROC_T *, struct nfsvattr *, int *, void *); int nfsrpc_write(vnode_t, struct uio *, int *, int *, - struct ucred *, NFSPROC_T *, struct nfsvattr *, int *, void *, int); + struct ucred *, NFSPROC_T *, struct nfsvattr *, int *, int, int); int nfsrpc_mknod(vnode_t, char *, int, struct vattr *, u_int32_t, enum vtype, struct ucred *, NFSPROC_T *, struct nfsvattr *, struct nfsvattr *, struct nfsfh **, int *, int *, void *); diff --git a/sys/fs/nfsclient/nfs.h b/sys/fs/nfsclient/nfs.h index ce1747a2ab6b..94faeb381ac0 100644 --- a/sys/fs/nfsclient/nfs.h +++ b/sys/fs/nfsclient/nfs.h @@ -107,7 +107,7 @@ void ncl_nodeunlock(struct nfsnode *); int ncl_getattrcache(struct vnode *, struct vattr *); int ncl_readrpc(struct vnode *, struct uio *, struct ucred *); int ncl_writerpc(struct vnode *, struct uio *, struct ucred *, int *, int *, - int); + int, int); int ncl_readlinkrpc(struct vnode *, struct uio *, struct ucred *); int ncl_readdirrpc(struct vnode *, struct uio *, struct ucred *, struct thread *); diff --git a/sys/fs/nfsclient/nfs_clbio.c b/sys/fs/nfsclient/nfs_clbio.c index 081aff6c0e7c..c89d6796ea00 100644 --- a/sys/fs/nfsclient/nfs_clbio.c +++ b/sys/fs/nfsclient/nfs_clbio.c @@ -793,7 +793,7 @@ do_sync: */ must_commit = 2; error = ncl_writerpc(vp, &uio, cred, &iomode, - &must_commit, 0); + &must_commit, 0, ioflag); KASSERT((must_commit == 2), ("ncl_directio_write: Updated write verifier")); if (error) @@ -986,11 +986,21 @@ ncl_write(struct vop_write_args *ap) * get the append lock. */ if (ioflag & IO_APPEND) { - np->n_attrstamp = 0; - KDTRACE_NFS_ATTRCACHE_FLUSH_DONE(vp); - error = VOP_GETATTR(vp, &vattr, cred); - if (error) - return (error); + /* + * For NFSv4, the AppendWrite will Verify the size against + * the file's size on the server. If not the same, the + * write will then be retried, using the file size returned + * by the AppendWrite. However, for NFSv2 and NFSv3, the + * size must be acquired here via a Getattr RPC. + * The AppendWrite is not done for a pNFS mount. + */ + if (!NFSHASNFSV4(nmp) || NFSHASPNFS(nmp)) { + np->n_attrstamp = 0; + KDTRACE_NFS_ATTRCACHE_FLUSH_DONE(vp); + error = VOP_GETATTR(vp, &vattr, cred); + if (error) + return (error); + } NFSLOCKNODE(np); uio->uio_offset = np->n_size; NFSUNLOCKNODE(np); @@ -1633,7 +1643,7 @@ ncl_doio_directwrite(struct buf *bp) * verifier on the mount point. */ must_commit = 2; - ncl_writerpc(bp->b_vp, uiop, bp->b_wcred, &iomode, &must_commit, 0); + ncl_writerpc(bp->b_vp, uiop, bp->b_wcred, &iomode, &must_commit, 0, 0); KASSERT((must_commit == 2), ("ncl_doio_directwrite: Updated write" " verifier")); if (iomode != NFSWRITE_FILESYNC) @@ -1818,7 +1828,7 @@ ncl_doio(struct vnode *vp, struct buf *bp, struct ucred *cr, struct thread *td, iomode = NFSWRITE_FILESYNC; error = ncl_writerpc(vp, uiop, cr, &iomode, &must_commit, - called_from_strategy); + called_from_strategy, 0); /* * When setting B_NEEDCOMMIT also set B_CLUSTEROK to try diff --git a/sys/fs/nfsclient/nfs_clrpcops.c b/sys/fs/nfsclient/nfs_clrpcops.c index f6b6e75b4278..64bcfbd4f943 100644 --- a/sys/fs/nfsclient/nfs_clrpcops.c +++ b/sys/fs/nfsclient/nfs_clrpcops.c @@ -134,7 +134,7 @@ static int nfsrpc_readrpc(vnode_t , struct uio *, struct ucred *, nfsv4stateid_t *, NFSPROC_T *, struct nfsvattr *, int *, void *); static int nfsrpc_writerpc(vnode_t , struct uio *, int *, int *, struct ucred *, nfsv4stateid_t *, NFSPROC_T *, struct nfsvattr *, int *, - void *); + int); static int nfsrpc_createv23(vnode_t , char *, int, struct vattr *, nfsquad_t, int, struct ucred *, NFSPROC_T *, struct nfsvattr *, struct nfsvattr *, struct nfsfh **, int *, int *, void *); @@ -1710,7 +1710,7 @@ nfsmout: int nfsrpc_write(vnode_t vp, struct uio *uiop, int *iomode, int *must_commit, struct ucred *cred, NFSPROC_T *p, struct nfsvattr *nap, int *attrflagp, - void *stuff, int called_from_strategy) + int called_from_strategy, int ioflag) { int error, expireret = 0, retrycnt, nostateid; u_int32_t clidrev = 0; @@ -1754,7 +1754,7 @@ nfsrpc_write(vnode_t vp, struct uio *uiop, int *iomode, int *must_commit, error = 0; else error = nfsrpc_writerpc(vp, uiop, iomode, must_commit, - newcred, &stateid, p, nap, attrflagp, stuff); + newcred, &stateid, p, nap, attrflagp, ioflag); if (error == NFSERR_STALESTATEID) nfscl_initiate_recovery(nmp->nm_clp); if (lckp != NULL) @@ -1789,18 +1789,19 @@ nfsrpc_write(vnode_t vp, struct uio *uiop, int *iomode, int *must_commit, static int nfsrpc_writerpc(vnode_t vp, struct uio *uiop, int *iomode, int *must_commit, struct ucred *cred, nfsv4stateid_t *stateidp, - NFSPROC_T *p, struct nfsvattr *nap, int *attrflagp, void *stuff) + NFSPROC_T *p, struct nfsvattr *nap, int *attrflagp, int ioflag) { u_int32_t *tl; struct nfsmount *nmp = VFSTONFS(vp->v_mount); struct nfsnode *np = VTONFS(vp); - int error = 0, len, tsiz, rlen, commit, committed = NFSWRITE_FILESYNC; - int wccflag = 0, wsize; + int error = 0, len, rlen, commit, committed = NFSWRITE_FILESYNC; + int wccflag = 0; int32_t backup; - struct nfsrv_descript nfsd; - struct nfsrv_descript *nd = &nfsd; + struct nfsrv_descript *nd; nfsattrbit_t attrbits; - off_t tmp_off; + uint64_t tmp_off; + ssize_t tsiz, wsize; + bool do_append; KASSERT(uiop->uio_iovcnt == 1, ("nfs: writerpc iovcnt > 1")); *attrflagp = 0; @@ -1812,14 +1813,31 @@ nfsrpc_writerpc(vnode_t vp, struct uio *uiop, int *iomode, return (EFBIG); } wsize = nmp->nm_wsize; + do_append = false; + if ((ioflag & IO_APPEND) != 0 && NFSHASNFSV4(nmp) && !NFSHASPNFS(nmp)) + do_append = true; NFSUNLOCKMNT(nmp); + nd = malloc(sizeof(*nd), M_TEMP, M_WAITOK); nd->nd_mrep = NULL; /* NFSv2 sometimes does a write with */ nd->nd_repstat = 0; /* uio_resid == 0, so the while is not done */ while (tsiz > 0) { *attrflagp = 0; len = (tsiz > wsize) ? wsize : tsiz; - NFSCL_REQSTART(nd, NFSPROC_WRITE, vp); + if (do_append) + NFSCL_REQSTART(nd, NFSPROC_APPENDWRITE, vp); + else + NFSCL_REQSTART(nd, NFSPROC_WRITE, vp); if (nd->nd_flag & ND_NFSV4) { + if (do_append) { + NFSZERO_ATTRBIT(&attrbits); + NFSSETBIT_ATTRBIT(&attrbits, NFSATTRBIT_SIZE); + nfsrv_putattrbit(nd, &attrbits); + NFSM_BUILD(tl, uint32_t *, 2 * NFSX_UNSIGNED + + NFSX_HYPER); + *tl++ = txdr_unsigned(NFSX_HYPER); + txdr_hyper(uiop->uio_offset, tl); tl += 2; + *tl = txdr_unsigned(NFSV4OP_WRITE); + } nfsm_stateidtom(nd, stateidp, NFSSTATEID_PUTSTATEID); NFSM_BUILD(tl, u_int32_t *, NFSX_HYPER+2*NFSX_UNSIGNED); txdr_hyper(uiop->uio_offset, tl); @@ -1878,9 +1896,11 @@ nfsrpc_writerpc(vnode_t vp, struct uio *uiop, int *iomode, *tl = txdr_unsigned(NFSV4OP_GETATTR); (void) nfsrv_putattrbit(nd, &attrbits); } - error = nfscl_request(nd, vp, p, cred, stuff); - if (error) + error = nfscl_request(nd, vp, p, cred, NULL); + if (error) { + free(nd, M_TEMP); return (error); + } if (nd->nd_repstat) { /* * In case the rpc gets retried, roll @@ -1895,11 +1915,33 @@ nfsrpc_writerpc(vnode_t vp, struct uio *uiop, int *iomode, } if (nd->nd_flag & (ND_NFSV3 | ND_NFSV4)) { error = nfscl_wcc_data(nd, vp, nap, attrflagp, - &wccflag, NULL); + &wccflag, &tmp_off); if (error) goto nfsmout; } + if ((nd->nd_flag & (ND_NFSV4 | ND_NOMOREDATA)) == + (ND_NFSV4 | ND_NOMOREDATA) && + nd->nd_repstat == NFSERR_NOTSAME && do_append) { + /* + * Verify of the file's size failed, so redo the + * write using the file's size as returned in + * the wcc attributes. + */ + if (tmp_off + tsiz <= nmp->nm_maxfilesize) { + do_append = false; + uiop->uio_offset = tmp_off; + m_freem(nd->nd_mrep); + nd->nd_mrep = NULL; + continue; + } else + nd->nd_repstat = EFBIG; + } if (!nd->nd_repstat) { + if (do_append) { + /* Strip off the Write reply status. */ + do_append = false; + NFSM_DISSECT(tl, uint32_t *, 2 * NFSX_UNSIGNED); + } if (nd->nd_flag & (ND_NFSV3 | ND_NFSV4)) { NFSM_DISSECT(tl, u_int32_t *, 2 * NFSX_UNSIGNED + NFSX_VERF); @@ -1964,6 +2006,7 @@ nfsmout: *iomode = committed; if (nd->nd_repstat && !error) error = nd->nd_repstat; + free(nd, M_TEMP); return (error); } diff --git a/sys/fs/nfsclient/nfs_clvnops.c b/sys/fs/nfsclient/nfs_clvnops.c index 38b637c5c062..a6040ce58290 100644 --- a/sys/fs/nfsclient/nfs_clvnops.c +++ b/sys/fs/nfsclient/nfs_clvnops.c @@ -1597,7 +1597,7 @@ ncl_readrpc(struct vnode *vp, struct uio *uiop, struct ucred *cred) */ int ncl_writerpc(struct vnode *vp, struct uio *uiop, struct ucred *cred, - int *iomode, int *must_commit, int called_from_strategy) + int *iomode, int *must_commit, int called_from_strategy, int ioflag) { struct nfsvattr nfsva; int error, attrflag, ret; @@ -1612,8 +1612,8 @@ ncl_writerpc(struct vnode *vp, struct uio *uiop, struct ucred *cred, NFSCL_DEBUG(4, "writerpc: aft doiods=%d\n", error); if (error != 0) error = nfsrpc_write(vp, uiop, iomode, must_commit, cred, - uiop->uio_td, &nfsva, &attrflag, NULL, - called_from_strategy); + uiop->uio_td, &nfsva, &attrflag, called_from_strategy, + ioflag); if (attrflag) { if (VTONFS(vp)->n_flag & ND_NFSV4) ret = nfscl_loadattrcache(&vp, &nfsva, NULL, NULL, 1,