Why is NFSv4 so slow?

[Rick Macklem]

On Mon, 28 Jun 2010, Rick C. Petty wrote:

> On Mon, Jun 28, 2010 at 12:35:14AM -0400, Rick Macklem wrote:
>>
>> Being stuck in "newnfsreq" means that it is trying to establish a TCP
>> connection with the server (again smells like some networking issue).
>> <snip>
>> Disabling delegations is the next step. (They aren't
>> required for correct behaviour and are disabled by default because
>> they are the "greenest" part of the implementation.)
>
> After disabling delegations, I was able to build world and kernel on two
> different clients, and my port build problems went away as well.
>

Ok, it sounds like you found some kind of race condition in the delegation
handling. (I'll see if I can reproduce it here. It could be fun to find:-)

> I'm still left with a performance problem, although not quite as bad as I
> originally reported.  Directory listings are snappy once again, but playing
> h264 video is choppy, particularly when seeking around: there's almost a
> full second delay before it kicks in, no matter where I seek.  With NFSv3
> the delay on seeks was less than 0.1 seconds and the playback was never
> jittery.
>

Hmm, see below w.r.t. 100% cpu.

> I can try it again with v3 client and v4 server, if you think that's
> worthy of pursuit.  If it makes any difference, the server's four CPUs are
> pegged at 100% (running "nice +4" cpu-bound jobs).  But that was the case
> before I enabled v4 server too.
>
It would be interesting to see if the performance problem exists for
NFSv3 mounts against the experimental (nfsv4) server.

Since the CPUs are 100% busy, it might be a scheduling issue w.r.t.
the nfsd threads (ie. the ones in the experimental server don't have
as high a priority as for the regular server?). I've always tested
on a machine where the CPU (I only have single core) are nowhere near
100% busy. If this theory is correct, the performance issue should
still be noticible for an NFSv3 mount to the experimental server.

I'll try running something compute bound on the server here and see
what happens.

rick