cvs commit: src/sys/nfsclient nfs.h nfs_socket.c nfs_subs.c
kris at FreeBSD.org
Fri Oct 12 13:46:03 PDT 2007
Mohan Srinivasan wrote:
> mohans 2007-10-12 19:12:21 UTC
> FreeBSD src repository
> Modified files:
> sys/nfsclient nfs.h nfs_socket.c nfs_subs.c nfsmount.h
> NFS MP scaling changes.
> - Eliminate the hideous nfs_sndlock that serialized NFS/TCP request senders
> thru the sndlock.
> - Institute a new nfs_connectlock that serializes NFS/TCP reconnects. Add
> logic to wait for pending request senders to finish sending before
> reconnecting. Dial down the sb_timeo for NFS/TCP sockets to 1 sec.
> - Break out the nfs xid manipulation under a new nfs xid lock, rather than
> over loading the nfs request lock for this purpose.
> - Fix some of the locking in nfs_request.
> Many thanks to Kris Kennaway for his help with this and for initiating the
> MP scaling analysis and work. Kris also tested this patch thorougly.
> Approved by: re@ (Ken Smith)
For the benefit of others: this change improved throughput by about 10%
at high I/O loads with a dual core client, and by a factor of 10 on an 8
core client (this was mostly the home-brew nfs_sndlock, which mohan
correctly describes :-).
Mohan's previous commit that increases the nfs server socket buffer size
is also very important for NFS performance. Without it I was only
getting 1-2MB/sec throughput over 10Gb ethernet with UDP mounts, because
the minuscule 32kb socket buffer was constantly filling up and forcing
retransmits. With the new default of 256KB I still get full buffers
with 10ge, so you may need to increase this further to eliminate this
problem. It might be OK for gige speeds, although I was still seeing
some buffer full events, so maybe we should consider increasing the
default sockbuf size to 512KB or so if this is widespread.
As a side comment there is a bug in either the nfs client or server that
corrupts I/O when there is packet loss with UDP mounts (the default).
TCP mounts handle this at the TCP layer.
More information about the cvs-src