Low nfs write throughput
John Baldwin
jhb at freebsd.org
Fri Nov 18 13:14:47 UTC 2011
On Thursday, November 17, 2011 10:10:27 pm Daryl Sayers wrote:
>
> Can anyone suggest why I am getting poor write performance from my nfs
setup.
> I have 2 x FreeBSD 8.2-STABLE i386 machines with ASUS P5B-plus mother
boards,
> 4G mem and Dual core 3g processor using 147G 15k Seagate SAS drives with
> onboard Gb network cards connected to an idle network. The results below
show
> that I get nearly 100Mb/s with a dd over rsh but only 15Mbs using nfs. It
> improves if I use async but a smbfs mount still beats it. I am using the
same
> file, source and destinations for all tests. I have tried alternate Network
> cards with no resulting benefit.
>
> oguido# dd if=/u0/tmp/D2 | rsh castor dd of=/dsk/ufs/D2
> 1950511+1 records in
> 1950511+1 records out
> 998661755 bytes transferred in 10.402483 secs (96002246 bytes/sec)
> 1950477+74 records in
> 1950511+1 records out
> 998661755 bytes transferred in 10.115458 secs (98726301 bytes/sec)
(98Mb/s)
>
>
> oguido# mount -t nfs -o wsize=65536,rsize=65536,tcp gemini:/dsk/ufs /mnt
> oguido# dd if=/u0/tmp/D2 of=/mnt/tmp/D2 bs=128k
> 7619+1 records in
> 7619+1 records out
> 998661755 bytes transferred in 62.570260 secs (15960646 bytes/sec)
(15Mb/s)
>
>
> oguido# mount -t nfs -o wsize=65536,rsize=65536,tcp,async gemini:/dsk/ufs
/mnt
> oguido# dd if=/u0/tmp/D2 of=/mnt/tmp/D2 bs=128k
> 7619+1 records in
> 7619+1 records out
> 998661755 bytes transferred in 50.697024 secs (19698627 bytes/sec)
(19Mb/s)
>
>
> oguido# mount -t smbfs //gemini/ufs /mnt
> oguido# dd if=/u0/tmp/D2 of=/mnt/tmp/D2 bs=128k
> 7619+1 records in
> 7619+1 records out
> 998661755 bytes transferred in 29.787616 secs (33526072 bytes/sec)
(33Mb/s)
>
> Looking at a systat -v on the destination I see that the nfs test does not
> exceed 16KB/t with 100% busy where the other tests reach up to 128KB/t.
> For the record I get reads of 22Mb/s without and 77Mb/s with async turned on
> for the nfs mount.
I don't know if it will help with your performance, but I have some patches
to allow the NFS server to cluster writes. You can try
www.freebsd.org/~jhb/patches/nfs_server_cluster.patch. I've tested it on 8,
but it should probably apply fine to 9.
--
John Baldwin
More information about the freebsd-stable
mailing list