[RFC] Should vfs.nfsrv.async be implemented for new NFS server?

[Bruce Evans]

On Sun, 6 Nov 2011, Josh Paetzel wrote:

> On 11/06/11 07:34, Rick Macklem wrote:
>> Ronald Klop wrote:
>>> On Sun, 06 Nov 2011 02:18:05 +0100, Rick Macklem
>>> <rmacklem at uoguelph.ca>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> Josh Paetzel pointed out that vfs.nfsrv.async doesn't exist
>>>> for the new NFS server.
>>>>
>>>> I don't think I had spotted this before, but when I looked I
>>>> saw that, when vfs.nfsrv.async is set non-zero in the old server,
>>>> it returns FILESYNC (which means the write has been committed to
>>>> non-volatile storage) even when it hasn't actually done that.
>>>>
>>>> This can improve performance, but has some negative implications:
>>>> ...
>>>
>>> Just out of curiosity. Why would lying about FILESYNC improve
>>> performance
>>> over UNSTABLE? The server does the same work. Only the client holds
>>> data
>>> longer in memory. I only see impact if the client has just a little
>>> bit of
>>> memory.
>>>
>>> Ronald.
>> Well, I'm not sure I have an answer. Josh noted that it makes a big
>> difference for them. Maybe he can elaborate?

It is not completely clear how fudging FILESYNC gives asyncness, but
in general asyncness improves performance by not being sync.  syncness
forces writing of tinygrams, often several times at the same place for
metadata.  asyncness should allow the server to delay writing for as
long as it wants.

> I'll test it out and report back in the next week or so.
>
> In 8.x, setting the async sysctl was the difference between 80-100MB/sec
> and 800 MB/sec (Yes, MegaBytes!) using a variety of different clients,
> including the VMWare ESXi 4.x client, Xen 5.6 client, various linux
> clients and the FreeBSD client.  I'll note that 800MB/sec is getting
> close to the underlying filesystem performance, so it's likely that the
> gate to performance is in the filesystem in that case.  80-100MB/sec is
> basically gigE performance.

Didn't you mount the client file systems async?  This has worked since
FreeBSD-3 or earlier.  It also helps to mount the server file system
async.

Unfortunately, this shows another need for the too-global sysctl on
the server -- non-FreeBSD clients might not support async mounts.

In my previous reply, I said that the sysctl might not be needed for
v2 clients, since even v2 clients can fudge the return code.  Now I
don't see how that can help -- v2 clients can't ask for UNSTABLE,
so v2 servers must always give them FILESYNC (or an error), so the
return code is already FILESYNC.

Bruce