From nobody Sun Aug 30 22:03:15 2026 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4hY5jN3T9Wz6qGCq for ; Sun, 30 Aug 2026 22:03:20 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "YR2" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4hY5jN3GSdz3T5b for ; Sun, 30 Aug 2026 22:03:20 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1788127400; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=jnUawusr/+1G45ktuubDLzgRy5J9diNSb21eaqUAecM=; b=IoUYvpDmFUNRrmPeAsrMiYvfen/Owx0f5QEWyEqkqy0sLfYPutMSOUjqCx7tZCsazkU2Gw 83xkarYDc6C+PCxObhpHW5EcuMLh2QOL/GF5gto9VlUGUMXAi6nQpZEMhpnCfaK2yNtSaV Txrg7Zh0sByNMOYyaRk8+fuvU8jtCf0Aqz+Xaenpluwcn1kL8sxOqwYLG3H/mrS5hWJdKg OgHplP6sERPuft/SBY6hk0TtC2iI4ZSh15wnFP7E2gKMEfaP46nFr0i3Mn8hpXgP0ZsuzA 8bMsjQi00zYb9G0SnyMwbUAsvEBZmwkvFoIVzYZjYeQBG1+18m/LHWWZH211Og== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1788127400; a=rsa-sha256; cv=none; b=BvM/XrrpRxPs86GCjyWL0ZHnphAWY61x3Tf5/OK79o5h0smO8XS3IWy+X0DVh3yGe7rN8s f8YVyqC2PVd6wpplpBHWsBQ0loELHi5MR7408IEIsnnNNuaQHB0L/7Pf0i1A6OVt8hMtij dv7FxG2VcFFduaY+yn+N7jOtWNlehzZyiBgxYn/u0RWPGWofMbhoU8GrTk1YUDmoWKXc4D Zyq8L9AsADouvRA4rJilx0eTgv+Pdpkszb9ApJOENwwgVSRV7nYEPLoENcgi7sQpQVYG0h v5v95zL0kTc1cxqCDnweFbmUJk7TQyprE0ii59ge1FLbheXYbGy3HfnqV710yA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1788127400; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=jnUawusr/+1G45ktuubDLzgRy5J9diNSb21eaqUAecM=; b=MjrkH2Ei5h0zHkGAOJ1xrId9Iv0zIThzbU3ULa7xLoqqfXgMtJ5GAXTAXdoeiBz4ruXLFy X7rxZIWC/PkTLs4DmpvihqIEyvPX9xih+mQwM4Dx30If5hnGmwygziZ7zS/frg+ctFqo9Q OyhNljcrY0fYPsFR4piYW9hdwu9B1jO5aOySQ2NlnwAXB4nRQjgjhni8ITLv9ZdYnMGTsm GdC2FPJe0AVnSLaDdv4TRD5C3wgXy9wImInNM7W5CMjDZsXj+ZxxrgZX+11TyJKBV7dwrJ gpzxhVmxfTfCdYP+899GeOZ8d+eVYxA/bACd7N0iI5lhiw80NW/iT1HE3LEFEA== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4hY5jN2CkHzkfJ for ; Sun, 30 Aug 2026 22:03:20 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 21be1 by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Sun, 30 Aug 2026 22:03:15 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Rick Macklem Subject: git: 884ee8d6c9b4 - main - nfscl: Add some glue for client side NFS over RDMA List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: rmacklem X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 884ee8d6c9b43b4c009f5eeda91162a0d0eb2bc7 Auto-Submitted: auto-generated Date: Sun, 30 Aug 2026 22:03:15 +0000 Message-Id: <6a94a8a3.21be1.2fd3eae5@gitrepo.freebsd.org> The branch main has been updated by rmacklem: URL: https://cgit.FreeBSD.org/src/commit/?id=884ee8d6c9b43b4c009f5eeda91162a0d0eb2bc7 commit 884ee8d6c9b43b4c009f5eeda91162a0d0eb2bc7 Author: Rick Macklem AuthorDate: 2026-08-30 22:00:27 +0000 Commit: Rick Macklem CommitDate: 2026-08-30 22:00:27 +0000 nfscl: Add some glue for client side NFS over RDMA This patch adds assorted bits needed by the nfsclrdma.ko module that implements client side NFS over RDMA. With this commit, the glue required by the nfsclrdma.ko module is complete and it should load ok. It should not affect non-RDMA operation. I've specified a long MFC, since the module still requires extensive testing and, hopefully, a review. MFC after: 3 months --- sys/fs/nfs/nfs_commonkrpc.c | 195 ++++++++++++++++++++++++---------------- sys/fs/nfs/nfsport.h | 2 + sys/fs/nfsclient/nfs_clrpcops.c | 75 +++++++++++++++- sys/fs/nfsclient/nfs_clvfsops.c | 124 +++++++++++++++++++++---- sys/fs/nfsclient/nfsmount.h | 2 + 5 files changed, 304 insertions(+), 94 deletions(-) diff --git a/sys/fs/nfs/nfs_commonkrpc.c b/sys/fs/nfs/nfs_commonkrpc.c index 705e65d81d41..813005ed0cc4 100644 --- a/sys/fs/nfs/nfs_commonkrpc.c +++ b/sys/fs/nfs/nfs_commonkrpc.c @@ -261,7 +261,7 @@ newnfs_connect(struct nfsmount *nmp, struct nfssockreq *nrp, CLIENT *client; struct netconfig *nconf; struct socket *so; - int one = 1, retries, error = 0; + int one = 1, retries, error = 0, val; struct thread *td = curthread; SVCXPRT *xprt; struct timeval timo; @@ -289,93 +289,101 @@ newnfs_connect(struct nfsmount *nmp, struct nfssockreq *nrp, td->td_ucred = cred; saddr = nrp->nr_nam; - if (saddr->sa_family == AF_INET) - if (nrp->nr_sotype == SOCK_DGRAM) + if (saddr->sa_family == AF_INET) { + if (nmp != NULL && NFSHASRDMA(nmp)) + nconf = getnetconfigent("rdma"); + else if (nrp->nr_sotype == SOCK_DGRAM) nconf = getnetconfigent("udp"); else nconf = getnetconfigent("tcp"); - else - if (nrp->nr_sotype == SOCK_DGRAM) + } else { + if (nmp != NULL && NFSHASRDMA(nmp)) + nconf = getnetconfigent("rdma6"); + else if (nrp->nr_sotype == SOCK_DGRAM) nconf = getnetconfigent("udp6"); else nconf = getnetconfigent("tcp6"); + } - pktscale = nfs_bufpackets; - if (pktscale < 2) - pktscale = 2; - if (pktscale > 64) - pktscale = 64; - pktscalesav = pktscale; - /* - * soreserve() can fail if sb_max is too small, so shrink pktscale - * and try again if there is an error. - * Print a log message suggesting increasing sb_max. - * Creating a socket and doing this is necessary since, if the - * reservation sizes are too large and will make soreserve() fail, - * the connection will work until a large send is attempted and - * then it will loop in the krpc code. - */ - so = NULL; - saddr = NFSSOCKADDR(nrp->nr_nam, struct sockaddr *); - error = socreate(saddr->sa_family, &so, nrp->nr_sotype, - nrp->nr_soproto, td->td_ucred, td); - if (error != 0) - goto out; - do { - if (error != 0 && pktscale > 2) { - if (nmp != NULL && nrp->nr_sotype == SOCK_STREAM && - pktscale == pktscalesav) { - /* - * Suggest vfs.nfs.bufpackets * maximum RPC message, - * adjusted for the sb_max->sb_max_adj conversion of - * MCLBYTES / (MSIZE + MCLBYTES) as the minimum setting - * for kern.ipc.maxsockbuf. - */ - tval = (NFS_MAXBSIZE + NFS_MAXXDR) * nfs_bufpackets; - tval *= MSIZE + MCLBYTES; - tval += MCLBYTES - 1; /* Round up divide by MCLBYTES. */ - tval /= MCLBYTES; - printf("Consider increasing kern.ipc.maxsockbuf to a " - "minimum of %ju to support %ubyte NFS I/O\n", - (uintmax_t)tval, NFS_MAXBSIZE); - } - pktscale--; - } - if (nrp->nr_sotype == SOCK_DGRAM) { - if (nmp != NULL) { - sndreserve = (NFS_MAXDGRAMDATA + NFS_MAXPKTHDR) * - pktscale; - rcvreserve = (NFS_MAXDGRAMDATA + NFS_MAXPKTHDR) * - pktscale; - } else { - sndreserve = rcvreserve = 1024 * pktscale; - } - } else { - if (nrp->nr_sotype != SOCK_STREAM) - panic("nfscon sotype"); - if (nmp != NULL) { - sndreserve = (NFS_MAXBSIZE + NFS_MAXXDR) * - pktscale; - rcvreserve = (NFS_MAXBSIZE + NFS_MAXXDR) * - pktscale; - } else { - sndreserve = rcvreserve = 1024 * pktscale; - } - } - error = soreserve(so, sndreserve, rcvreserve); - if (error != 0 && nmp != NULL && nrp->nr_sotype == SOCK_STREAM && - pktscale <= 2) - printf("Must increase kern.ipc.maxsockbuf or reduce" - " rsize, wsize\n"); - } while (error != 0 && pktscale > 2); - soclose(so); + sndreserve = rcvreserve = 0; + if (nmp == NULL || !NFSHASRDMA(nmp)) { + pktscale = nfs_bufpackets; + if (pktscale < 2) + pktscale = 2; + if (pktscale > 64) + pktscale = 64; + pktscalesav = pktscale; + /* + * soreserve() can fail if sb_max is too small, so shrink + * pktscale and try again if there is an error. + * Print a log message suggesting increasing sb_max. + * Creating a socket and doing this is necessary since, if the + * reservation sizes are too large and will make soreserve() + * fail, the connection will work until a large send is + * attempted and then it will loop in the krpc code. + */ + so = NULL; + saddr = NFSSOCKADDR(nrp->nr_nam, struct sockaddr *); + error = socreate(saddr->sa_family, &so, nrp->nr_sotype, + nrp->nr_soproto, td->td_ucred, td); + if (error != 0) + goto out; + do { + if (error != 0 && pktscale > 2) { + if (nmp != NULL && nrp->nr_sotype == SOCK_STREAM && + pktscale == pktscalesav) { + /* + * Suggest vfs.nfs.bufpackets * maximum RPC message, + * adjusted for the sb_max->sb_max_adj conversion of + * MCLBYTES / (MSIZE + MCLBYTES) as the minimum + * setting for kern.ipc.maxsockbuf. + */ + tval = (NFS_MAXBSIZE + NFS_MAXXDR) * nfs_bufpackets; + tval *= MSIZE + MCLBYTES; + tval += MCLBYTES - 1; /* Round up divide. */ + tval /= MCLBYTES; + printf("Consider increasing kern.ipc.maxsockbuf to " + "a minimum of %ju to support %ubyte NFS I/O\n", + (uintmax_t)tval, NFS_MAXBSIZE); + } + pktscale--; + } + if (nrp->nr_sotype == SOCK_DGRAM) { + if (nmp != NULL) { + sndreserve = (NFS_MAXDGRAMDATA + + NFS_MAXPKTHDR) * pktscale; + rcvreserve = (NFS_MAXDGRAMDATA + + NFS_MAXPKTHDR) * pktscale; + } else { + sndreserve = rcvreserve = 1024 * pktscale; + } + } else { + if (nrp->nr_sotype != SOCK_STREAM) + panic("nfscon sotype"); + if (nmp != NULL) { + sndreserve = (NFS_MAXBSIZE + NFS_MAXXDR) * + pktscale; + rcvreserve = (NFS_MAXBSIZE + NFS_MAXXDR) * + pktscale; + } else { + sndreserve = rcvreserve = 1024 * pktscale; + } + } + error = soreserve(so, sndreserve, rcvreserve); + if (error != 0 && nmp != NULL && + nrp->nr_sotype == SOCK_STREAM && pktscale <= 2) + printf("Must increase kern.ipc.maxsockbuf or reduce" + " rsize, wsize\n"); + } while (error != 0 && pktscale > 2); + soclose(so); + } if (error != 0) goto out; client = clnt_reconnect_create(nconf, saddr, nrp->nr_prog, nrp->nr_vers, sndreserve, rcvreserve); CLNT_CONTROL(client, CLSET_WAITCHAN, "nfsreq"); - if (nmp != NULL) { + if (nmp != NULL && !NFSHASRDMA(nmp)) { if ((nmp->nm_flag & NFSMNT_INT)) CLNT_CONTROL(client, CLSET_INTERRUPTIBLE, &one); if ((nmp->nm_flag & NFSMNT_RESVPORT)) @@ -466,7 +474,7 @@ newnfs_connect(struct nfsmount *nmp, struct nfssockreq *nrp, retries = nfs_dsretries; } } - } else { + } else if (nmp == NULL) { /* * Three cases: * - Null RPC callback to client @@ -481,6 +489,41 @@ newnfs_connect(struct nfsmount *nmp, struct nfssockreq *nrp, } if (dotls) CLNT_CONTROL(client, CLSET_TLS, &one); + } else { + /* RDMA. */ + if (NFSHASNFSV4N(nmp) && cred != NULL) { + /* + * Set up the backchannel. svc_vc_create_backchannel() + * is sufficient. The rdma boolean in conn_cf will get + * set by the CLSET_BACKCHANNEL control. + */ + /* + * Make sure the nfscbd_pool doesn't get + * destroyed while doing this. + */ + NFSD_LOCK(); + if (nfs_numnfscbd > 0) { + nfs_numnfscbd++; + NFSD_UNLOCK(); + xprt = svc_vc_create_backchannel(nfscbd_pool); + CLNT_CONTROL(client, CLSET_BACKCHANNEL, xprt); + NFSD_LOCK(); + nfs_numnfscbd--; + if (nfs_numnfscbd == 0) + wakeup(&nfs_numnfscbd); + } + NFSD_UNLOCK(); + } + + /* Set the small reply size for a Readdir. */ + val = NFS_DIRBLKSIZ + PAGE_SIZE; + CLNT_CONTROL(client, CLSET_RDMASMALL_REPLY, &val); + val = NFSV4_CBSLOTS; + CLNT_CONTROL(client, CLSET_RDMA_CBSLOTS, &val); + if (NFSHASSOFT(nmp)) + retries = nmp->nm_retry; + else + retries = INT_MAX; } CLNT_CONTROL(client, CLSET_RETRIES, &retries); diff --git a/sys/fs/nfs/nfsport.h b/sys/fs/nfs/nfsport.h index 91345bde3441..cd50d12dbe5f 100644 --- a/sys/fs/nfs/nfsport.h +++ b/sys/fs/nfs/nfsport.h @@ -1065,6 +1065,8 @@ void ncl_copy_vattr(struct vnode *vp, struct vattr *dst, struct vattr *src); (n)->nm_minorvers > 0) #define NFSHASTLS(n) (((n)->nm_newflag & NFSMNT_TLS) != 0) #define NFSHASSYSKRB5(n) (((n)->nm_newflag & NFSMNT_SYSKRB5) != 0) +#define NFSHASRDMA(n) (((n)->nm_newflag & NFSMNT_RDMA) != 0) +#define NFSHASNOWRITEREDUCE(n) (((n)->nm_newflag & NFSMNT_NOWRITEREDUCE) != 0) /* * Set boottime. diff --git a/sys/fs/nfsclient/nfs_clrpcops.c b/sys/fs/nfsclient/nfs_clrpcops.c index 7984737a2f65..6e84567649d7 100644 --- a/sys/fs/nfsclient/nfs_clrpcops.c +++ b/sys/fs/nfsclient/nfs_clrpcops.c @@ -49,6 +49,7 @@ #include #include #include +#include SYSCTL_DECL(_vfs_nfs); @@ -236,6 +237,8 @@ static int nfsrpc_seekrpc(vnode_t, off_t *, nfsv4stateid_t *, bool *, int, struct nfsvattr *, int *, struct ucred *); static struct mbuf *nfsm_split(struct mbuf *, uint64_t); static void nfscl_statfs(struct vnode *, struct ucred *, NFSPROC_T *); +static struct mbuf *nfsm_build_rdma_reduction(struct nfsrv_descript *nd, + int len, int pos, bool to_mem); int nfs_pnfsio(task_fn_t *, void *); @@ -1692,11 +1695,16 @@ nfsrpc_readlink(vnode_t vp, struct uio *uiop, struct ucred *cred, u_int32_t *tl; struct nfsrv_descript nfsd, *nd = &nfsd; struct nfsnode *np = VTONFS(vp); + struct nfsmount *nmp; nfsattrbit_t attrbits; int error, len, cangetattr = 1; *attrflagp = 0; + nmp = VFSTONFS(vp->v_mount); NFSCL_REQSTART(nd, NFSPROC_READLINK, vp, cred); + /* For RDMA, mark that a one page rdma_reply is required. */ + if (NFSHASRDMA(nmp)) + nd->nd_mreq->m_flags |= M_PROTO7; if (nd->nd_flag & ND_NFSV4) { /* * And do a Getattr op. @@ -1817,13 +1825,18 @@ nfsrpc_readrpc(vnode_t vp, struct uio *uiop, struct ucred *cred, u_int32_t *tl; int error = 0, len, retlen, tsiz, eof = 0; struct nfsrv_descript nfsd; + struct mbuf *mr; struct nfsmount *nmp = VFSTONFS(vp->v_mount); struct nfsrv_descript *nd = &nfsd; int rsize; off_t tmp_off; + bool did_rdma; *attrflagp = 0; tsiz = uiop->uio_resid; + did_rdma = false; + if (NFSHASRDMA(nmp) && tsiz > RPCRDMA_MAX_SMALL_MSG / 2) + did_rdma = true; tmp_off = uiop->uio_offset + tsiz; NFSLOCKMNT(nmp); if (tmp_off > nmp->nm_maxfilesize || tmp_off < uiop->uio_offset) { @@ -1848,6 +1861,9 @@ nfsrpc_readrpc(vnode_t vp, struct uio *uiop, struct ucred *cred, txdr_hyper(uiop->uio_offset, tl); *(tl + 2) = txdr_unsigned(len); } + /* For RDMA, make the data a separate chunk. */ + if (did_rdma) + mr = nfsm_build_rdma_reduction(nd, len, 0, true); /* * Since I can't do a Getattr for NFSv4 for Write, there * doesn't seem any point in doing one here, either. @@ -1876,7 +1892,12 @@ nfsrpc_readrpc(vnode_t vp, struct uio *uiop, struct ucred *cred, eof = fxdr_unsigned(int, *tl); } NFSM_STRSIZ(retlen, len); - error = nfsm_mbufuio(nd, uiop, retlen); + if (!did_rdma) { + error = nfsm_mbufuio(nd, uiop, retlen); + } else { + error = rpc_copy_uio_pages(mr, uiop, retlen, true); + rpc_free_rdma_reduction(mr); + } if (error) goto nfsmout; m_freem(nd->nd_mrep); @@ -1998,14 +2019,18 @@ nfsrpc_writerpc(vnode_t vp, struct uio *uiop, int *iomode, int wccflag = 0; int32_t backup; struct nfsrv_descript *nd; + struct mbuf *mr; nfsattrbit_t attrbits; uint64_t tmp_off; ssize_t tsiz, wsize; - bool do_append; + bool do_append, did_rdma; KASSERT(uiop->uio_iovcnt == 1, ("nfs: writerpc iovcnt > 1")); *attrflagp = 0; tsiz = uiop->uio_resid; + did_rdma = false; + if (NFSHASRDMA(nmp) && tsiz > RPCRDMA_MAX_SMALL_MSG / 2) + did_rdma = true; tmp_off = uiop->uio_offset + tsiz; NFSLOCKMNT(nmp); if (tmp_off > nmp->nm_maxfilesize || tmp_off < uiop->uio_offset) { @@ -2069,7 +2094,16 @@ nfsrpc_writerpc(vnode_t vp, struct uio *uiop, int *iomode, *tl++ = x; /* total to this offset */ *tl = x; /* size of this write */ } - error = nfsm_uiombuf(nd, uiop, len); + /* For RDMA, make the data a separate chunk. */ + if (did_rdma && !NFSHASNOWRITEREDUCE(nmp)) { + rlen = m_length(nd->nd_mreq, NULL); + mr = nfsm_build_rdma_reduction(nd, len, rlen, false); + error = rpc_copy_uio_pages(mr, uiop, len, false); + if (error) + rpc_free_rdma_reduction(mr); + } else { + error = nfsm_uiombuf(nd, uiop, len); + } if (error != 0) { m_freem(nd->nd_mreq); free(nd, M_TEMP); @@ -2106,6 +2140,8 @@ nfsrpc_writerpc(vnode_t vp, struct uio *uiop, int *iomode, free(nd, M_TEMP); return (error); } + if (did_rdma && !NFSHASNOWRITEREDUCE(nmp)) + rpc_free_rdma_reduction(mr); if (nd->nd_repstat) { /* * In case the rpc gets retried, roll @@ -3631,6 +3667,9 @@ nfsrpc_readdir(vnode_t vp, struct uio *uiop, nfsuint64 *cookiep, while (more_dirs && bigenough) { *attrflagp = 0; NFSCL_REQSTART(nd, NFSPROC_READDIR, vp, cred); + /* For RDMA, mark that a rdma_reply is needed. */ + if (NFSHASRDMA(nmp)) + nd->nd_mreq->m_flags |= M_PROTO8; if (nd->nd_flag & ND_NFSV2) { NFSM_BUILD(tl, u_int32_t *, 2 * NFSX_UNSIGNED); *tl++ = cookie.lval[1]; @@ -4122,6 +4161,9 @@ nfsrpc_readdirplus(vnode_t vp, struct uio *uiop, nfsuint64 *cookiep, while (more_dirs && bigenough) { *attrflagp = 0; NFSCL_REQSTART(nd, NFSPROC_READDIRPLUS, vp, cred); + /* For RDMA, mark that a small rdma_reply is needed. */ + if (NFSHASRDMA(nmp)) + nd->nd_mreq->m_flags |= M_PROTO8; NFSM_BUILD(tl, u_int32_t *, 6 * NFSX_UNSIGNED); *tl++ = cookie.lval[0]; *tl++ = cookie.lval[1]; @@ -5372,6 +5414,9 @@ nfsrpc_getacl(struct vnode *vp, acl_type_t acltype, struct ucred *cred, (acltype == ACL_TYPE_ACCESS || acltype == ACL_TYPE_DEFAULT)) return (EOPNOTSUPP); NFSCL_REQSTART(nd, NFSPROC_GETACL, vp, cred); + /* For RDMA, mark that a large rdma_reply is required. */ + if (NFSHASRDMA(nmp)) + nd->nd_mreq->m_flags |= M_PROTO9; NFSZERO_ATTRBIT(&attrbits); if (acltype == ACL_TYPE_NFS4) NFSSETBIT_ATTRBIT(&attrbits, NFSATTRBIT_ACL); @@ -9581,6 +9626,9 @@ nfsrpc_getextattr(vnode_t vp, const char *name, struct uio *uiop, ssize_t *lenp, *attrflagp = 0; NFSCL_REQSTART(nd, NFSPROC_GETEXTATTR, vp, cred); + /* For RDMA, mark that a large rdma_reply is required. */ + if (NFSHASRDMA(VFSTONFS(vp->v_mount))) + nd->nd_mreq->m_flags |= M_PROTO9; nfsm_strtom(nd, name, strlen(name)); NFSM_BUILD(tl, uint32_t *, NFSX_UNSIGNED); *tl = txdr_unsigned(NFSV4OP_GETATTR); @@ -10047,3 +10095,24 @@ nfscl_statfs(struct vnode *vp, struct ucred *cred, NFSPROC_T *td) mtx_unlock(&nmp->nm_mtx); } } + +/* + * Set up the RDMA reduction mbuf in the build list. + */ +static struct mbuf * +nfsm_build_rdma_reduction(struct nfsrv_descript *nd, int len, int pos, + bool to_mem) +{ + struct mbuf *m, *mr; + + mr = rpc_reduce_pg(len, pos, to_mem); + nd->nd_mb->m_next = mr; + nd->nd_mb = mr; + NFSMCLGET(m, M_NOWAIT); + m->m_len = 0; + nd->nd_bpos = mtod(m, char *); + nd->nd_mb->m_next = m; + nd->nd_mb = m; + nd->nd_mreq->m_flags |= M_PROTO11; + return (mr); +} diff --git a/sys/fs/nfsclient/nfs_clvfsops.c b/sys/fs/nfsclient/nfs_clvfsops.c index 74e5e2dc9b1b..e057324f1f20 100644 --- a/sys/fs/nfsclient/nfs_clvfsops.c +++ b/sys/fs/nfsclient/nfs_clvfsops.c @@ -38,6 +38,8 @@ #include "opt_bootp.h" #include "opt_nfsroot.h" #include "opt_kern_tls.h" +#include "opt_inet.h" +#include "opt_inet6.h" #include #include @@ -74,6 +76,7 @@ #include #include +#include #include FEATURE(nfscl, "NFSv4 client"); @@ -87,6 +90,7 @@ extern struct nfsmount *ncl_iodmount[NFS_MAXASYNCDAEMON]; extern struct mtx ncl_iod_mutex; NFSCLSTATEMUTEX; extern struct mtx nfsrv_dslock_mtx; +extern int newnfs_directio_enable; MALLOC_DEFINE(M_NEWNFSREQ, "newnfsclient_req", "NFS request header"); MALLOC_DEFINE(M_NEWNFSMNT, "newnfsmnt", "NFS mount struct"); @@ -192,6 +196,7 @@ int newnfs_iosize(struct nfsmount *nmp) { int iosize, maxio; + uint32_t rdma_maxio; /* First, set the upper limit for iosize */ if (nmp->nm_flag & NFSMNT_NFSV4) { @@ -204,6 +209,15 @@ newnfs_iosize(struct nfsmount *nmp) } else { maxio = NFS_V2MAXDATA; } + + /* For an RDMA mount, find out what the RDMA's limit is. */ + if (NFSHASRDMA(nmp) && nmp->nm_sockreq.nr_client != NULL) { + CLNT_CONTROL(nmp->nm_sockreq.nr_client, CLGET_RDMAMAX_IO, + &rdma_maxio); + if (rdma_maxio > 0) + maxio = MIN(maxio, rdma_maxio); + } + if (nmp->nm_rsize > maxio || nmp->nm_rsize == 0) nmp->nm_rsize = maxio; if (nmp->nm_rsize > NFS_MAXBSIZE) @@ -228,6 +242,11 @@ newnfs_iosize(struct nfsmount *nmp) iosize = imax(iosize, PAGE_SIZE); iosize = imax(iosize, NFS_DIRBLKSIZ); nmp->nm_mountp->mnt_stat.f_iosize = iosize; + if (NFSHASRDMA(nmp) && nmp->nm_sockreq.nr_client != NULL) { + rdma_maxio = iosize; + CLNT_CONTROL(nmp->nm_sockreq.nr_client, CLSET_RDMAMAX_IO, + &rdma_maxio); + } return (iosize); } @@ -788,7 +807,7 @@ static const char *nfs_opts[] = { "from", "nfs_args", "nfsv3", "sec", "principal", "nfsv4", "gssname", "allgssname", "dirpath", "minorversion", "nametimeo", "negnametimeo", "nocto", "noncontigwr", "pnfs", "wcommitsize", "oneopenown", "tls", "tlscertname", "nconnect", - "syskrb5", NULL }; + "syskrb5", "rdma", "nowritereduce", NULL }; /* * Parse the "from" mountarg, passed by the generic mount(8) program @@ -939,7 +958,7 @@ nfs_mount(struct mount *mp) krbnamelen, srvkrbnamelen; size_t hstlen; uint32_t newflag; - int aconn = 0; + int aconn = 0, rdma_port; has_nfs_args_opt = 0; has_nfs_from_opt = 0; @@ -1031,6 +1050,8 @@ nfs_mount(struct mount *mp) args.flags |= NFSMNT_ONEOPENOWN; if (vfs_getopt(mp->mnt_optnew, "tls", NULL, NULL) == 0) newflag |= NFSMNT_TLS; + if (vfs_getopt(mp->mnt_optnew, "nowritereduce", NULL, NULL) == 0) + newflag |= NFSMNT_NOWRITEREDUCE; if (vfs_getopt(mp->mnt_optnew, "tlscertname", (void **)&opt, &len) == 0) { /* @@ -1247,6 +1268,27 @@ nfs_mount(struct mount *mp) } if (vfs_getopt(mp->mnt_optnew, "syskrb5", NULL, NULL) == 0) newflag |= NFSMNT_SYSKRB5; + if (vfs_getopt(mp->mnt_optnew, "rdma", (void **)&opt, NULL) == + 0) { + ret = sscanf(opt, "%d", &rdma_port); + if (ret != 1 || rdma_port < 1 || rdma_port > IPPORT_MAX) { + vfs_mount_error(mp, "Bad RDMA port#: %s", opt); + error = EINVAL; + goto out; + } + if (PMAP_HAS_DMAP == 0) { + vfs_mount_error(mp, "RDMA requires a DMAP"); + error = EINVAL; + goto out; + } + if (clnt_rdma_create_call == NULL) { + vfs_mount_error(mp, "RDMA requires the nfsclrdma.ko " + "module be loaded"); + error = EINVAL; + goto out; + } + newflag |= NFSMNT_RDMA; + } if (vfs_getopt(mp->mnt_optnew, "sec", (void **) &secname, NULL) == 0) nfs_sec_name(secname, &args.flags); @@ -1414,19 +1456,26 @@ nfs_mount(struct mount *mp) } } - if (aconn > 0 && (args.sotype != SOCK_STREAM || - (args.flags & NFSMNT_NFSV4) == 0 || minvers == 0)) { - /* - * RFC 5661 requires that an NFSv4.1/4.2 server - * send an RPC reply on the same TCP connection - * as the one it received the request on. - * This property in required for "nconnect" and - * might not be the case for NFSv3 or NFSv4.0 servers. - */ - vfs_mount_error(mp, "nconnect should only be used " - "for NFSv4.1/4.2 mounts"); - error = EINVAL; - goto out; + if (aconn > 0) { + if (args.sotype != SOCK_STREAM || + (args.flags & NFSMNT_NFSV4) == 0 || minvers == 0) { + /* + * RFC 5661 requires that an NFSv4.1/4.2 server + * send an RPC reply on the same TCP connection + * as the one it received the request on. + * This property in required for "nconnect" and + * might not be the case for NFSv3 or NFSv4.0 servers. + */ + vfs_mount_error(mp, "nconnect should only be used " + "for NFSv4.1/4.2 mounts"); + error = EINVAL; + goto out; + } else if ((newflag & NFSMNT_RDMA) != 0) { + vfs_mount_error(mp, "nconnect cannot be used " + "for RDMA mounts"); + error = EINVAL; + goto out; + } } if ((newflag & NFSMNT_SYSKRB5) != 0 && @@ -1462,6 +1511,47 @@ nfs_mount(struct mount *mp) goto out; } + if ((newflag & NFSMNT_RDMA) != 0) { +#ifdef INET + struct sockaddr_in *sin; +#endif +#ifdef INET6 + struct sockaddr_in6 *sin6; +#endif + + if (rdma_check_route == NULL) { + vfs_mount_error(mp, "nfsclrdma module not loaded"); + error = EINVAL; + goto out; + } + + /* Replace the NFS port# with the NFS RDMA one. */ + switch (nam->sa_family) { +#ifdef INET + case AF_INET: + sin = (struct sockaddr_in *)nam; + sin->sin_port = htons(rdma_port); + break; +#endif +#ifdef INET6 + case AF_INET6: + sin6 = (struct sockaddr_in6 *)nam; + sin6->sin6_port = htons(rdma_port); + break; +#endif + default: + vfs_mount_error(mp, "rdma address not inet/inet6"); + error = EINVAL; + goto out; + } + error = rdma_check_route(vnet0, nam, NFSV4_CBSLOTS); + if (error != 0) { + vfs_mount_error(mp, "rdma is not configured"); + error = EINVAL; + goto out; + } + } + args.fh = nfh; error = mountnfs(&args, mp, nam, hst, krbname, krbnamelen, dirpath, dirlen, srvkrbname, srvkrbnamelen, &vp, td->td_ucred, td, @@ -2238,6 +2328,10 @@ void nfscl_retopts(struct nfsmount *nmp, char *buffer, size_t buflen) &blen); nfscl_printopt(nmp, (nmp->nm_newflag & NFSMNT_SYSKRB5) != 0, ",syskrb5", &buf, &blen); + nfscl_printopt(nmp, (nmp->nm_newflag & NFSMNT_RDMA) != 0, + ",rdma", &buf, &blen); + nfscl_printopt(nmp, (nmp->nm_newflag & NFSMNT_NOWRITEREDUCE) != 0, + ",nowritereduce", &buf, &blen); nfscl_printopt(nmp, (nmp->nm_flag & NFSMNT_NOCONN) != 0, ",noconn", &buf, &blen); nfscl_printoptval(nmp, nmp->nm_aconnect + 1, ",nconnect", &buf, &blen); diff --git a/sys/fs/nfsclient/nfsmount.h b/sys/fs/nfsclient/nfsmount.h index ef876dd30e59..800aaeb7f46d 100644 --- a/sys/fs/nfsclient/nfsmount.h +++ b/sys/fs/nfsclient/nfsmount.h @@ -129,6 +129,8 @@ struct nfsmount { /* New mount flags only used by the kernel via nmount(2). */ #define NFSMNT_TLS 0x00000001 #define NFSMNT_SYSKRB5 0x00000002 +#define NFSMNT_RDMA 0x00000004 +#define NFSMNT_NOWRITEREDUCE 0x00000008 #define NFSMNT_DIRPATH(m) (&((m)->nm_name[(m)->nm_krbnamelen + 1])) #define NFSMNT_SRVKRBNAME(m) \