[RFC] Should vfs.nfsrv.async be implemented for new NFS server?

[Rick Macklem]

Ivan Voras wrote:
> On 06/11/2011 02:18, Rick Macklem wrote:
> > Hi,
> >
> > Josh Paetzel pointed out that vfs.nfsrv.async doesn't exist
> > for the new NFS server.
> >
> > I don't think I had spotted this before, but when I looked I
> > saw that, when vfs.nfsrv.async is set non-zero in the old server,
> > it returns FILESYNC (which means the write has been committed to
> > non-volatile storage) even when it hasn't actually done that.
> 
> Do I understand this correctly: the server normally (for async=0) does
> a
> fsync after any writes and returns FILESYNC status to the client?
> 
No, the server only does an "fsync" if the client specifies FILESYNC.

In normal (async==0) writes are done UNSTABLE (which means the
server has the data but has not yet committed it to stable storage like
a disk), unless the client specifies FILESYNC.
The server then replies UNSTABLE to the client, so the client
knows that it must keep the modified data in its cache until it does
a Commit RPC.
When async!=0, the only difference is that the server lies
and replies FILESYNC, even though it hasn't done that.

> This seems too extreme... doesn't NFSv4 have its own fsync()-like RPC
> that does that manually? If it does, then I don't think there are any
> differences between doing a write() on a local file system with e.g.
> soft-updates enabled and doing a write on a NFS file system - in both
> cases, no data is even remotely guaranteed to survive a crash unless a
> fsync (or equivalent operation) was issued.
> 
The handling of writes is essentially the same for NFSv3 and NFSv4.
The client requests writes be either FILESYNC (everything committed
to stable storage before the server replies) or UNSTABLE (where the
data is only cached in the server and could be lost if the server
crashes). For the UNSTABLE case, the client must do a subsequent
Commit RPC (which can cover all UNSTABLE writes for a file and is
essentially an fsync) before assuming the data is safely stored in
the server.
(NFSv4.1 adds the optional capability of pNFS, where the client can do I/O against
a separate Data Server, but we aren't there yet.)

The trouble with "async!=0" is that a Commit RPC may never happen, since
the server has lied and told the client it doesn't need to do one. Most
servers will commit changes to stable storage sooner or later, but there
is no guarantee that is the case.

> On 06/11/2011 17:25, Josh Paetzel wrote:
> > In 8.x, setting the async sysctl was the difference between
> > 80-100MB/sec
> > and 800 MB/sec (Yes, MegaBytes!) using a variety of different
> > clients,
> 
> Yup, this is in any case too big not to add the async mode.

Ok, I understand this as a vote for committing it. If that isn't the
case, please email again, clarifying it.

Thanks, rick