Low nfs write throughput

Daryl Sayers daryl at ci.com.au
Tue Nov 29 00:12:44 UTC 2011


>>>>> "Bengt" == Bengt Ahlgren <bengta at sics.se> writes:

> Daryl Sayers <daryl at ci.com.au> writes:
>> Can anyone suggest why I am getting poor write performance from my nfs setup.
>> I have 2 x FreeBSD 8.2-STABLE i386 machines with ASUS P5B-plus mother boards,
>> 4G mem and Dual core 3g processor using 147G 15k Seagate SAS drives with
>> onboard Gb network cards connected to an idle network. The results below show
>> that I get nearly 100Mb/s with a dd over rsh but only 15Mbs using nfs. It
>> improves if I use async but a smbfs mount still beats it. I am using the same
>> file, source and destinations for all tests. I have tried alternate Network
>> cards with no resulting benefit.

> [...]

>> Looking at a systat -v on the destination I see that the nfs test does not
>> exceed 16KB/t with 100% busy where the other tests reach up to 128KB/t.
>> For the record I get reads of 22Mb/s without and 77Mb/s with async turned on
>> for the nfs mount.

> On an UFS filesystem you get NFS writes with the same size as the
> filesystem blocksize.  So an easy way to improve performance is to
> create a filesystem with larger blocks.  I accidentally found this out
> when I had two NFS exported filesystems from the same box with 16K and
> 64K blocksizes respectively.

> (Larger blocksize also tremendously improves the performance of UFS
> snapshots!)

Thanks to all that answered. I did try the 'sysctl -w vfs.nfsrv.async=1' with
no reportable change in performance. We are using a UFS2 filesystem so the
zfs command was not required. I did not try the patch as we would like to stay
as standard as possible but will upgrade if the patch is released in new
kernel.
Thanks Bengt for the suggestion of block size. Increasing the block size to
64k made a significant improvement to performance.

-- 
Daryl Sayers                             Direct: +612 95525510
Corinthian Engineering                   Office: +612 95525500
Suite 54, Jones Bay Wharf                   Fax: +612 95525549
26-32 Pirrama Rd                          email: daryl at ci.com.au
Pyrmont NSW 2009 Australia                  www: http://www.ci.com.au


More information about the freebsd-stable mailing list