Why is NFSv4 so slow?

Rick Macklem rmacklem at uoguelph.ca
Sun Aug 29 15:44:08 UTC 2010


> Hi. I'm still having problems with NFSv4 being very laggy on one
> client.
> When the NFSv4 server is at 50% idle CPU and the disks are < 1% busy,
> I am
> getting horrible throughput on an idle client. Using dd(1) with 1 MB
> block
> size, when I try to read a > 100 MB file from the client, I'm getting
> around 300-500 KiB/s. On another client, I see upwards of 20 MiB/s
> with
> the same test (on a different file). On the broken client:
> 

Since other client(s) are working well, that seems to suggest that it
is a network related problem and not a bug in the NFS code.

First off, the obvious question: How does this client differ from the
one that performs much better?
Do they both use the same "re" network interface for the NFS traffic?
(If the answer is "no", I'd be suspicious that the "re" hardware or
device driver is the culprit.)

Things that I might try in an effort to isolate the problem:
- switch the NFS traffic to use the nfe0 net interface.
- put a net interface identical to the one on the client that
  works well in the machine and use that for the NFS traffic.
- turn off TXCSUM and RXCSUM on re0
- reduce the read/write data size, using rsize=N,wsize=N on the
  mount. (It will default to MAXBSIZE and some net interfaces don't
  handle large bursts of received data well. If you drop it to
  rsize=8192,wszie=8192 and things improve, then increase N until it
  screws up.)
- check the port configuration on the switch end, to make sure it
  is also 1000bps-full duplex.
- move the client to a different net port on the switch or even a
  different switch (and change the cable, while you're at it).
- Look at "netstat -s" and see if there are a lot of retransmits
  going on in TCP.

If none of the above seems to help, you could look at a packet trace
and see what is going on. Look for TCP reconnects (SYN, SYN-ACK...)
or places where there is a large time delay/retransmit of a TCP
segment.

Hopefully others who are more familiar with the networking side
can suggest other things to try, rick



More information about the freebsd-stable mailing list