Weird Linux - FreeBSD/ZFS NFSv4 interoperability problem

[Rick Macklem]

> > A couple of people have reported very slow read rates for the NFSv4
> > client (actually the experimental client, since they see it for
> > NFSv3 too). If you could easily do the following, using a FreeBSD8.1
> > or newer client:
> > # mount -t nfs -o nfsv4 <any-server>:/path <mnt-path>
> > - cd to anywhere in the mount that has a 100Mbyte+ file
> > # dd if=<100Mbyte+ file> of=/dev/null bs=1m
> >
> > and then report what read rate you see along with the client's
> > machine-arch/# of cores/ram size/network driver used by the mount
> >
> > rick
> > ps: Btw, anyone else who can do this test, it would be appreciated.
> >    If you aren't set up for NFSv4, you can do an NFSv3 mount using
> >    the exp. client instead.
> >    # mount -t newnfs -o nfsv3 <any-server>:/path <mnt-path>
> 
> On 8-STABLE (both client and server). First test is NFSv3 on the
> standard
> client:
> 
> (0:842) new-gate:~terry# mount -t nfs -o nfsv4 new-rz1:/data /foo
> [tcp6] new-rz1:/data: NFSPROC_NULL: RPC: Program/version mismatch; low
> version = 2, high version = 3
> [tcp] new-rz1:/data: NFSPROC_NULL: RPC: Program/version mismatch; low
> version = 2, high version = 3
> 
> ^C
> (1:843) new-gate:~terry# mount -t nfs -o nfsv3 new-rz1:/data /foo
> [...]
> (0:869) new-gate:/foo/Backups/Suzanne VAIO# dd if=0cff3d7b_VOL.spf
> of=/dev/null bs=1m
> 6010+1 records in
> 6010+1 records out
> 6301945344 bytes transferred in 69.730064 secs (90376302 bytes/sec)
> 
> Now, let's try the newnfs client (cache should have been primed by the
> first run, so we'd expect this to be faster):
> 

Just thought I'd mention that, since it is a different mount, the caches
won't be primed, which is good, because that would mask differences.

> (0:879) new-gate:/tmp# umount /foo
> (0:880) new-gate:/tmp# mount -t newnfs -o nfsv3 new-rz1:/data /foo
> (0:881) new-gate:/tmp# cd /foo/Backups/Suzanne\ VAIO/
> (0:882) new-gate:/foo/Backups/Suzanne VAIO# dd if=0cff3d7b_VOL.spf
> of=/dev/null bs=1m
> 6010+1 records in
> 6010+1 records out
> 6301945344 bytes transferred in 135.927222 secs (46362644 bytes/sec)
> 
> Hmmm. Half the performance. The problem isn't the disk speed on the
> server:
> 

Ok, good. You aren't seeing what the two guys reported (they were really
slow, at less than 2Mbytes/sec). If you would like to, you could try the
following, since the two clients use different default r/w sizes.

# mount -t newnfs -o nfsv3,rsize=32768,wsize=32768 new-rz1:/data /foo

and see how it changes the read rate. I don't know why there is a
factor of 2 difference (if it isn't the different r/w size), but it
will probably get resolved as I bring the experimental client up to date.

Thanks a lot for doing the test and giving me a data point, rick