From nobody Wed Nov 04 20:52:48 2026 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4jKd1X2PHpz701J1 for ; Wed, 04 Nov 2026 20:52:48 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4jKd1X1MSWz4Q2S for ; Wed, 04 Nov 2026 20:52:48 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1793825568; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=jpZ3dHsGD4zNc7hmD+H+/t7k/LLA2RQtPjCkmETRZs4=; b=m6PNRvbH0XLpfxJH80KAWMyaA8q4J9bYrS6kgmiUj6/tLr8veNVkRpidoHi2JNqoqOhXu9 9dXhZwNaLgCzTHlKlMLBFAbmX6ZXA8DYJqAmkXQPlWBkrOJ1ahGEJc3QWWnj/K+O3BEFF1 BTAs96m8OcZEZPICumZfH9VU/izjYiqQs3JlM3NJzFKK7mg/j/GZDG9AWsYoHESkOYzS3B 7R51nihtUPXUGHtS8XLvolZAP48xn43IRqU30kATwNrw5Zq1K1ywKRZs3LGG4k33+bPFSY Q2hJ2NWQF2DEpRukoEb7uRKrquigcXbPpamUtQW/a+cm7DBQoPC4lvNlh2whyw== ARC-Seal: i=1; a=rsa-sha256; d=freebsd.org; s=dkim; cv=none; t=1793825568; b=Fe9fyQrmJUV33jojeSTDYjHF1JWmiqMJ0rYxSwse1UUuW4RNO3+/4coKUb1lyunBWVXids nAzhO6fF2tPfu/lH7kv8SLVGpkgs5nCGPQ9T++do7jmWFrvp9ni7twQJ52JTon988Mhdsn /sIFYn2GbvfRJ+74lUlu/lt9dDdItbyQNqzPXx2Xz5HWhr7vLxnZIOmDHxkTW3wlOjHOAa A0nQLbvNqseRBpgXMjPAJvxspnJPdBZ/Fon0vc9zPpe2h4DrxT/joBFizNAAbzypJWIitV P0wLNp32zfMiWwLuVpfpl5pCyNVuV6X+ZwnbonNwJXJ4kirsjoPeQ9sslUwBDA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1793825568; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=jpZ3dHsGD4zNc7hmD+H+/t7k/LLA2RQtPjCkmETRZs4=; b=CgsmB912Nf2zyxfSmprV8W4puykq/6gVYfVVZFnTrBrj6PYvA4gZTXHadHZ4Ac4ZqSRDan qLFKqeTY5cQmLR6mxhxaa/XyskypUSLwUu9jcoR3efVnFM93TuCl/YcY/qGUjZRXk1/QmK gYl0CgXwNb3wqmx8dz0PieDIedllg8IIbfPUlXkjqXCUCWjHC0NvYeRZ+rH3B30RgyrSKc jM/QvXFSOEPKglkBHfffTugVBB+g4aEPh8evSXKumvtwqlNFdcSGDa8hR+klqnvA1T4OK8 5poGMy2+WPkMSFOPiUzhRcnMnL9QqF0QWqWtCUt5dDLhDUkKvarjniZBCG+Thw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4jKd1X12HvzJtG for ; Wed, 04 Nov 2026 20:52:48 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 1d507 by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Wed, 04 Nov 2026 20:52:48 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Rick Macklem Subject: git: db8444c1b7e4 - main - nfscl: Fix the RDMA write reduction code List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org List-Id: Commit messages for all branches of the src repository List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: rmacklem X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: db8444c1b7e4cfe3df105268fa963a93c3780b02 Auto-Submitted: auto-generated Date: Wed, 04 Nov 2026 20:52:48 +0000 Message-Id: <6aeb9b20.1d507.32e31402@gitrepo.freebsd.org> The branch main has been updated by rmacklem: URL: https://cgit.FreeBSD.org/src/commit/?id=db8444c1b7e4cfe3df105268fa963a93c3780b02 commit db8444c1b7e4cfe3df105268fa963a93c3780b02 Author: Rick Macklem AuthorDate: 2026-11-04 20:50:48 +0000 Commit: Rick Macklem CommitDate: 2026-11-04 20:50:48 +0000 nfscl: Fix the RDMA write reduction code The write reduction code was broken in a few ways: - It didn't handle non-zero b_dirtyoff. - It didn't advance the uiop fields such that they could be backed up for a retry. - It didn't handle the rare retry case with a partially written buffer. This patch fixes the above and now write reduction seems to be working ok. MFC after: 1 month Fixes: 884ee8d6c9b4 ("nfscl: Add some glue for client side NFS over RDMA") --- sys/fs/nfsclient/nfs_clbio.c | 9 +++++++-- sys/fs/nfsclient/nfs_clrpcops.c | 29 ++++++++++++++++++++++------- sys/rpc/rpc_generic.c | 15 +++++++-------- 3 files changed, 36 insertions(+), 17 deletions(-) diff --git a/sys/fs/nfsclient/nfs_clbio.c b/sys/fs/nfsclient/nfs_clbio.c index f26da2598c45..d46691e83a35 100644 --- a/sys/fs/nfsclient/nfs_clbio.c +++ b/sys/fs/nfsclient/nfs_clbio.c @@ -1693,8 +1693,13 @@ ncl_doio(struct vnode *vp, struct buf *bp, struct ucred *cr, struct thread *td, else iomode = NFSWRITE_FILESYNC; - error = ncl_writerpc(vp, uiop, bp->b_pages, cr, &iomode, - &must_commit, called_from_strategy, 0); + if ((bp->b_dirtyoff % PAGE_SIZE) == 0) + error = ncl_writerpc(vp, uiop, + &bp->b_pages[bp->b_dirtyoff / PAGE_SIZE], cr, + &iomode, &must_commit, called_from_strategy, 0); + else + error = ncl_writerpc(vp, uiop, NULL, cr, &iomode, + &must_commit, called_from_strategy, 0); /* * When setting B_NEEDCOMMIT also set B_CLUSTEROK to try diff --git a/sys/fs/nfsclient/nfs_clrpcops.c b/sys/fs/nfsclient/nfs_clrpcops.c index 1f6227e0831a..395bf2ef74c3 100644 --- a/sys/fs/nfsclient/nfs_clrpcops.c +++ b/sys/fs/nfsclient/nfs_clrpcops.c @@ -1779,6 +1779,8 @@ nfsrpc_read(vnode_t vp, struct uio *uiop, vm_page_t *pgp, struct ucred *cred, (void)nfscl_getstateid(vp, nfhp->nfh_fh, nfhp->nfh_len, NFSV4OPEN_ACCESSREAD, 0, newcred, p, &stateid, &lckp); + if (retrycnt > 0) + pgp = NULL; error = nfsrpc_readrpc(vp, uiop, pgp, newcred, &stateid, p, nap, attrflagp); if (error == NFSERR_OPENMODE) { @@ -1832,8 +1834,8 @@ nfsrpc_readrpc(vnode_t vp, struct uio *uiop, vm_page_t *pgp, struct ucred *cred, struct nfsmount *nmp = VFSTONFS(vp->v_mount); struct nfsrv_descript *nd = &nfsd; int rsize; - off_t tmp_off; - bool did_rdma, reduced, first_loop; + off_t tmp_off, start_off; + bool did_rdma, reduced; *attrflagp = 0; tsiz = uiop->uio_resid; @@ -1846,7 +1848,7 @@ nfsrpc_readrpc(vnode_t vp, struct uio *uiop, vm_page_t *pgp, struct ucred *cred, rsize = nmp->nm_rsize; NFSUNLOCKMNT(nmp); nd->nd_mrep = NULL; - first_loop = true; + start_off = uiop->uio_offset; while (tsiz > 0) { mr = NULL; *attrflagp = 0; @@ -1859,8 +1861,9 @@ nfsrpc_readrpc(vnode_t vp, struct uio *uiop, vm_page_t *pgp, struct ucred *cred, * within the pages, this can only be done for the first * iteration. */ - if (first_loop && pgp != NULL && - (uiop->uio_offset & PAGE_MASK) == 0 && + tmp_off = start_off - uiop->uio_offset; + if (pgp != NULL && (tmp_off & PAGE_MASK) == 0 && + (start_off & PAGE_MASK) == 0 && uiop->uio_segflg == UIO_SYSSPACE && !NFSHASNOREADREDUCE(nmp)) did_rdma = true; @@ -1871,7 +1874,6 @@ nfsrpc_readrpc(vnode_t vp, struct uio *uiop, vm_page_t *pgp, struct ucred *cred, else mbflag = M_PROTO9; } - first_loop = false; NFSCL_REQSTART(nd, NFSPROC_READ, vp, cred); nd->nd_mreq->m_flags |= mbflag; if (nd->nd_flag & ND_NFSV4) @@ -1887,7 +1889,8 @@ nfsrpc_readrpc(vnode_t vp, struct uio *uiop, vm_page_t *pgp, struct ucred *cred, } /* For RDMA, make the data a separate chunk. */ if (did_rdma) - mr = nfsm_build_rdma_reduction(pgp, nd, len, 0, true); + mr = nfsm_build_rdma_reduction(&pgp[tmp_off/PAGE_SIZE], + nd, len, 0, true); /* * Since I can't do a Getattr for NFSv4 for Write, there * doesn't seem any point in doing one here, either. @@ -1920,6 +1923,11 @@ nfsrpc_readrpc(vnode_t vp, struct uio *uiop, vm_page_t *pgp, struct ucred *cred, } NFSM_STRSIZ(retlen, len); if (retlen > 0) { + /* + * Since nfsm_build_rdma_reduction() didn't allocate + * pages, the call to rpc_copy_uio_pages() only + * updates the uiop fields. It does not copy data. + */ if (!did_rdma || !reduced) error = nfsm_mbufuio(nd, uiop, retlen); else @@ -1995,6 +2003,8 @@ nfsrpc_write(vnode_t vp, struct uio *uiop, vm_page_t *pgp, int *iomode, } } + if (retrycnt > 0) + pgp = NULL; /* * If there is no stateid for NFSv4, it means this is an * extraneous write after close. Basically a poorly @@ -2137,6 +2147,11 @@ nfsrpc_writerpc(vnode_t vp, struct uio *uiop, vm_page_t *pgp, int *iomode, rlen = m_length(nd->nd_mreq, NULL); mr = nfsm_build_rdma_reduction(pgp, nd, len, rlen, false); + /* + * Since nfsm_build_rdma_reduction() didn't allocate + * pages, the call to rpc_copy_uio_pages() only + * updates the uiop fields. It does not copy data. + */ error = rpc_copy_uio_pages(mr, uiop, len, false); } else { error = nfsm_uiombuf(nd, uiop, len); diff --git a/sys/rpc/rpc_generic.c b/sys/rpc/rpc_generic.c index 15aa357b1775..8b923fb52067 100644 --- a/sys/rpc/rpc_generic.c +++ b/sys/rpc/rpc_generic.c @@ -1008,6 +1008,10 @@ rpc_copy_uio_pages(struct mbuf *mr, struct uio *uiop, int siz, bool from_pages) while (siz > 0) { if (uiop->uio_iovcnt <= 0 || uiop->uio_iov == NULL) return (EBADRPC); + if (uiop->uio_iov->iov_len == 0) { + uiop->uio_iovcnt--; + uiop->uio_iov++; + } left = uiop->uio_iov->iov_len; uiocp = uiop->uio_iov->iov_base; if (left > siz) @@ -1055,14 +1059,9 @@ rpc_copy_uio_pages(struct mbuf *mr, struct uio *uiop, int siz, bool from_pages) uiop->uio_offset += xfer; uiop->uio_resid -= xfer; } - if (uiop->uio_iov->iov_len <= siz) { - uiop->uio_iovcnt--; - uiop->uio_iov++; - } else { - uiop->uio_iov->iov_base = (void *) - ((char *)uiop->uio_iov->iov_base + uiosiz); - uiop->uio_iov->iov_len -= uiosiz; - } + uiop->uio_iov->iov_base = (void *) + ((char *)uiop->uio_iov->iov_base + uiosiz); + uiop->uio_iov->iov_len -= uiosiz; siz -= uiosiz; } return (0);