From nobody Thu Jun 09 04:22:05 2022 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 13DA68507F1; Thu, 9 Jun 2022 04:22:06 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4LJWC16qSPz3RLf; Thu, 9 Jun 2022 04:22:05 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1654748526; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=3X1JOjpYL0gBIGNhr7GxwUSTslQm0oRyTQ42tSV09pQ=; b=QAtb7grPs1khCKIwV9klZvCCJ2MH4kcF5rZQHFneHXJtzeCGWPkbTnLikYf2MWLgAZnoJ4 aU8bXr8h6pVKY85lQ+g64PBAdacaYMa0z1dmemmlGvJttl1zI4nZpZwHXnwNQbucC4WgF4 uILEYE19W1hrD0iQTEm3qIeR7IfEokgHJtwyEAatND6amJrGp4uvjRz1dTo65uyBga8Msj fBclQWfeYQBTrTceJEQrTjsNn0zBJU12q2jxbVwjyPZ7KaLzN9sis2E61NDoBEws3X7LoD nBTEYmwq9Bdb7Sssr2mUId60xe1eVTNLvEd/i6yh4qqYEgqlO4NL96RRLJmTgQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id CAC1D1D497; Thu, 9 Jun 2022 04:22:05 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 2594M52R008719; Thu, 9 Jun 2022 04:22:05 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 2594M5Df008718; Thu, 9 Jun 2022 04:22:05 GMT (envelope-from git) Date: Thu, 9 Jun 2022 04:22:05 GMT Message-Id: <202206090422.2594M5Df008718@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Rick Macklem Subject: git: ebea872f8915 - stable/13 - nfscl: Always invalidate buffers for append writes List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: rmacklem X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: ebea872f891567ea8610cb8e00acc443b59dbe0d Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1654748526; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=3X1JOjpYL0gBIGNhr7GxwUSTslQm0oRyTQ42tSV09pQ=; b=WlFMm27IOTagN2Ao0JAWIV2blTQkS/q3gTdai5E4Gp/kmaJhBVtaH85GtvNSZiiNuCKuCC 2lArJTYz3jMvnioA8yCdzqWWju8FxyoVCAuRdZsA1A00dh6B1MDSC8s8+SMsCD46pJGAiS sQ7lt1fnWf0+ucZWgtTCKEK3QzsV5VB1u0xP56Xa3Ofuv3qkoBE/HcUUyepsK2EawU6h81 YoPLWoW3uFQdIdtxibRRDFGJZIRPYSu1zS9iYU7FmP/3Y79o7Jv82MPjCDs3MPrLo5PfXP mt4XQh6rWziTWfqCVLP9ngYC4KDXS2HMT73e0+QHJ6IFbYJSP/nZmeNICm4BLA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1654748526; a=rsa-sha256; cv=none; b=suXZb683nZQAI0e8LrhpLUsdMHOgh3ThqpC7erFw3kSX39nnbxqcNF8rudbM8awOabwoW3 X7+hqz/RIale+TdpEO93Pg8A7yieqGWzWtAsgasCPHN7LeHndHwu0uoHjN8dbu6XqkiyuY MclqLQJ69C1PoJB0t7wwD3IjI0cW3Ua48aLeouxdDjm6a0wZ3vDyN89/ofu55Ip9eFQTHP GnTV56ogzHNeRssJPcqOudOM4kBEh0PJX45ZCNn3/FFBnn8M/2JOJ1i00UXQW7j2X0bJ0/ 1PcqDOFwlFufSKTW/Pv5qxaKAsJrh5COqe0EhvGbJbfDCxerlYSdwhmJRkICWQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch stable/13 has been updated by rmacklem: URL: https://cgit.FreeBSD.org/src/commit/?id=ebea872f891567ea8610cb8e00acc443b59dbe0d commit ebea872f891567ea8610cb8e00acc443b59dbe0d Author: Rick Macklem AuthorDate: 2022-01-06 22:18:36 +0000 Commit: Rick Macklem CommitDate: 2022-06-09 04:20:42 +0000 nfscl: Always invalidate buffers for append writes kib@ reported a problem which was resolved by reverting commit 867c27c23a5c, which changed the NFS client to use direct RPCs to the server for IO_APPEND writes. He also spotted that the code only invalidated buffer cache buffers when they were marked NMODIFIED (had been written into). This patch modifies the NFS VOP_WRITE() to always invalidate the buffer cache buffers and pages for the file when IO_APPEND is specified. It also includes some cleanup suggested by kib@. (cherry picked from commit e4df1036f66dc360606fea053ec04ccabb69df14) --- sys/fs/nfsclient/nfs_clbio.c | 67 +++++++++++++++++++++++++++----------------- 1 file changed, 41 insertions(+), 26 deletions(-) diff --git a/sys/fs/nfsclient/nfs_clbio.c b/sys/fs/nfsclient/nfs_clbio.c index 908f59855730..081aff6c0e7c 100644 --- a/sys/fs/nfsclient/nfs_clbio.c +++ b/sys/fs/nfsclient/nfs_clbio.c @@ -947,27 +947,33 @@ ncl_write(struct vop_write_args *ap) * Synchronously flush pending buffers if we are in synchronous * mode or if we are appending. */ - if (ioflag & (IO_APPEND | IO_SYNC)) { - NFSLOCKNODE(np); - if (np->n_flag & NMODIFIED) { - NFSUNLOCKNODE(np); -#ifdef notyet /* Needs matching nonblock semantics elsewhere, too. */ - /* - * Require non-blocking, synchronous writes to - * dirty files to inform the program it needs - * to fsync(2) explicitly. - */ - if (ioflag & IO_NDELAY) - return (EAGAIN); -#endif - np->n_attrstamp = 0; - KDTRACE_NFS_ATTRCACHE_FLUSH_DONE(vp); - error = ncl_vinvalbuf(vp, V_SAVE | ((ioflag & - IO_VMIO) != 0 ? V_VMIO : 0), td, 1); - if (error != 0) - return (error); - } else - NFSUNLOCKNODE(np); + if ((ioflag & IO_APPEND) || ((ioflag & IO_SYNC) && (np->n_flag & + NMODIFIED))) { + /* + * For the case where IO_APPEND is being done using a + * direct output (to the NFS server) RPC and + * newnfs_directio_enable is 0, all buffer cache buffers, + * including ones not modified, must be invalidated. + * This ensures that stale data is not read out of the + * buffer cache. The call also invalidates all mapped + * pages and, since the exclusive lock is held on the vnode, + * new pages cannot be faulted in. + * + * For the case where newnfs_directio_enable is set + * (which is not the default), it is not obvious that + * stale data should be left in the buffer cache, but + * the code has been this way for over a decade without + * complaints. Note that, unlike doing IO_APPEND via + * a direct write RPC when newnfs_directio_enable is not set, + * when newnfs_directio_enable is set, reading is done via + * direct to NFS server RPCs as well. + */ + np->n_attrstamp = 0; + KDTRACE_NFS_ATTRCACHE_FLUSH_DONE(vp); + error = ncl_vinvalbuf(vp, V_SAVE | ((ioflag & + IO_VMIO) != 0 ? V_VMIO : 0), td, 1); + if (error != 0) + return (error); } orig_resid = uio->uio_resid; @@ -998,11 +1004,20 @@ ncl_write(struct vop_write_args *ap) if (uio->uio_resid == 0) return (0); - if (vp->v_type == VREG && ((newnfs_directio_enable && (ioflag & - IO_DIRECT)) || (ioflag & IO_APPEND))) { - if ((ioflag & IO_APPEND) != 0) - ioflag |= IO_SYNC; - return nfs_directio_write(vp, uio, cred, ioflag); + /* + * Do IO_APPEND writing via a synchronous direct write. + * This can result in a significant performance improvement. + */ + if ((newnfs_directio_enable && (ioflag & IO_DIRECT)) || + (ioflag & IO_APPEND)) { + /* + * Direct writes to the server must be done NFSWRITE_FILESYNC, + * because the write data is not cached and, therefore, the + * write cannot be redone after a server reboot. + * Set IO_SYNC to make this happen. + */ + ioflag |= IO_SYNC; + return (nfs_directio_write(vp, uio, cred, ioflag)); } /*