Major issues with nfsv4

[Rick Macklem]

J David wrote:
>On Sun, Dec 13, 2020 at 4:25 PM Konstantin Belousov <kostikbel at gmail.com> wrote:
>> Nullfs with -o nocache (default for NFS mounts) should not cache vnodes.
>> So it is more likely a local load that has 130k files open.  Of course,
>> it is the OP who can answer the question.
>
>This I can rule out; there is no visible correlation between "Opens"
>and the number of files open on the system.
>
>Just finishing a test right now, and:
>
>$ sudo nfsstat -E -c | fgrep -A1 OpenOwner
>    OpenOwner        Opens    LockOwner        Locks       Delegs     LocalOwn
>         4678        36245           15            6            0            0
>$ sudo fstat  | wc -l
>    2698
>$ ps Haxlww | wc -l
>    1012
>
>The value of Opens increases consistently over time.
>
>Killing the processes causing this behavior *did not* reduce the
>number of OpenOwner or Opens.
>
>Unmounting the nullfs mounts (after the processes were gone) *did*:
>
>$ sudo nfsstat -E -c | fgrep -A1 OpenOwner
>    OpenOwner        Opens    LockOwner        Locks       Delegs     LocalOwn
>          130           41            0            0            0            0
Iwill take a look at the nullfs code to see if I can spot anything that might
explain this.

>Mutex contention was observed this time, but once it was apparent that
>"Opens" was increasing over time, I didn't let the test get to the
>point of disrupting activities.  This test ended at Opens = 36589,
>which is well short of the previous 130,000+.  It is possible that
>mutex contention becomes an issue once system CPU resources are
>exhausted.
>
>More about the results of the latest test after the data is analyzed.
>
>After that's done, I'll attempt Rick's patch.
The patch should help w.r.t. lock contention, but if the Open
count is increasing, you'll "hit the wall eventually".

>  In the long run, we
>would definitely like to get delegation to work.  Baby steps!
Well, all delegations do is allow a file to be repeatedly opened/closed
without talking to the server. But they are per-file and issued on the
first open against the server. As such, they are only useful if processes
are opening/closing the same file over and over again.
-->They also add complexity and, as such, they are often a loss and
      not a help. They are disabled on the FreeBSD NFS server by default
      and disabled on the client side unless the nfscbd(8) is running.

rick

Thanks!