NFSv3, ZFS, 10GE performance

[Sven Brandenburg]

On 03/26/2012 11:47 PM, Rick Macklem wrote:
> MAX_BSIZE is 64kb. I'd like to try making that bigger, but haven't gotten
> around to it yet. (If you wanted to try bumping MAX_BSIZE to 128Kb on both
> client and server and seeing what happens, that might be interesting, since
> my understanding is that ZFS uses a 128Kb block size.)

I finally came around and tested it (with 256k and 1M) - there is good 
and bad news.
The good news is the system does indeed boot (off of zfs at least, no 
idea on ufs) and it does increase performance.
I am now seeing roughly 800MB/s off the bat which is quite nice.
The bad news is that I had to use a Linux client because the FreeBSD 
client declined to work:
mount_nfs: /mnt, : No buffer space available

(Although I will freely admit that my knowledge of where to ajdust this 
value is rather limited: What I did was changing MAXBSIZE MAXPHYS to 1M 
in /usr/src/sys/sys/param.h, remaking world+kernel then reboot.
I forgot MAXPHYS in my first try and this crashed the client machine as 
soon as I tried to mount something via nfs. Notably, the server seems to 
be working ok even with a mismatched MAXPHYS/MAXBSIZE).

So far, the results are very promising.

regards,
Sven