Bad performance of 7.0 nfs client with Solaris nfs server
alfred at freebsd.org
Thu Feb 21 05:11:56 UTC 2008
* Kris Kennaway <kris at FreeBSD.org> [080220 13:42] wrote:
> Chuck Swiger wrote:
> >On Feb 20, 2008, at 1:01 PM, Alfred Perlstein wrote:
> >>>Take a look at the level of packet fragmentation you are encountering;
> >>>yes, this is expected and things will work but there is extra latency
> >>>added when the IP stack has to reassemble packets before the data can
> >>>be delivered. Try setting the NFS rsize/wsize to 1024 or perhaps 1400
> >>>and see whether that improves performance.
> >>>Or, if your switch and NICs support it, see whether you can get Gb
> >>>Ethernet jumbo frames working so that you don't have to fragment for
> >>>2K or 4K data packets....
> >>TCP mounts do not have this problem. You can safely use
> >>32k or higher sizes with TCP without fragmentation.
> >Oh, sure. But there is a bit more overhead with TCP transport than
> >UDP-- for local (switched) networks, UDP generally seems to be a
> >win...TCP seems to be a better choice over a VPN or some similar kind of
> Actually this is no longer true. At modern LAN speeds (e.g. gige) you
> can transmit packets fast enough that two things happen:
> 1) UDP socket buffer overruns are common, leading to packet loss.
> 2) the 16-bit sequence numbers wrap *much* faster than the IP fragment
> lifetime (30 seconds).
> These combine to cause data corruption when fragmented packets are
> dropped and then reassembled with a later transmission of the same
> sequence number.
> TCP mounts should be used whenever possible thesedays (I flipped the
> default mode in 8.0 the other day).
Additionally with smaller read/write sizes you must generate more
RPCs, for instance using a tcp read size of 32k will allow the
client to generate a single NFS_READ request for that 32k, however
a UDP mount at 1.2k will need to generate appoximately 26 requests
and the server will have to service that many requests as well.
- Alfred Perlstein
More information about the freebsd-performance