Why is NFSv4 so slow?

[Rick C. Petty]

On Mon, Sep 13, 2010 at 11:15:34AM -0400, Rick Macklem wrote:
> > 
> > instead of from the local cache. I also made sure that
> > the file was in the cache on the server, so the server's
> > disk speed is irrelevant.
> > 

<snip>

> > So, nfs is roughly twice as fast as newnfs, indeed.

Hmm, I have the same network switch as Oliver, and I wasn't caching the
file on the server before.  When I cache the file on the server, I get
about 1 MiB/s faster throughput, so that doesn't seem to make the
difference to me (but with higher throughputs, I would imagine it would).

> Thanks for doing the test. I think I can find out what causes the
> factor of 2 someday. What is really weird is that some people see
> several orders of magnitude slower (a few Mbytes/sec).
> 
> Your case was also useful, because you are using the same net
> interface/driver as the original report of a few Mbytes/sec, so it
> doesn't appear to be an re problem.

I believe I said something to that effect.  :-P

The problem I have is that the magnitude of throughput varies randomly.
Sometimes I can repeat the test and see 3-4 MB/s.  Then my server's
motherboard failed last week so I swapped things around and now I have 9-10
MB/s on the same client (but using 100Mbit interface instead of gigabit, so
those speeds make sense).

One thing I noticed is the lag seems to have disappeared after the reboots.
Another thing I had to change was that I was using an NFSv3 mount for /home
(with the v3 client, not the experimental v3/v4 client) and now I'm using
NFSv4 mounts exclusively.  Too much hardware changed because of that board
failing (AHCI was randomly dropping disks, and it got to the point that it
wouldn't pick up drives after a cold start and then the board failed to
POST 11 of 12 times), so I haven't been able to reliably reproduce any
problems.  I also had to reboot the "bad" client because of the broken
NFSv3 mountpoints, and the server was auto-upgraded to a newer 8.1-stable
(I often run "make buildworld kernel" regularly, so any reboots will
automatically have a newer kernel).

There's definite evidence that the newnfs mounts are slower than plain nfs,
and sometimes orders of magnitude slower (as others have shown).  But the
old nfs is so broken in other ways that I'd prefer slower yet more stable.
Thanks again for all your help, Rick!

-- Rick C. Petty