NFSv3, ZFS, 10GE performance

[Sven Brandenburg]

On 03/26/2012 12:37 PM, Ivan Voras wrote:
> You could try modifying the rsize and wsize NFS options (read
> mount_nfs(8)), they help with UFS.

I tried this a few days ago and fiddling rsize alters performance from 
"ok" to "terrible".
However, you made me revisit this and mount_nfs(8) seems to have a gem 
in its options: readahead.
This did the trick for me and my (long and sequential) reads.
While the manpage says its limited to 0-4, the best results were 
achieved with readahead=8 : 1.1GB/s - which is what I had hoped for.

On a tangent: gnu-dd 1GB/s is 10^9 Bytes/s, not 2^30. Yes, I fell for it 
at first :)
The good news is that there was no fiddling on the NFS server side. 
(Apart from MTU increases, PCI settings and more buffers to get TCP 
performance to full tilt in the first place)

Hopefully, readahead doesn't kill performance for smaller files.. :-)

regards,
Sven