Weird Linux - FreeBSD/ZFS NFSv4 interoperability problem

[Josh Paetzel]

On Sunday 12 September 2010 10:28:08 Rick Macklem wrote:
> > > A couple of people have reported very slow read rates for the NFSv4
> > > client (actually the experimental client, since they see it for
> > > NFSv3 too). If you could easily do the following, using a FreeBSD8.1
> > > or newer client:
> > > # mount -t nfs -o nfsv4 <any-server>:/path <mnt-path>
> > > - cd to anywhere in the mount that has a 100Mbyte+ file
> > > # dd if=<100Mbyte+ file> of=/dev/null bs=1m
> > > 
> > > and then report what read rate you see along with the client's
> > > machine-arch/# of cores/ram size/network driver used by the mount
> > > 
> > > rick
> > > ps: Btw, anyone else who can do this test, it would be appreciated.
> > > 
> > >    If you aren't set up for NFSv4, you can do an NFSv3 mount using
> > >    the exp. client instead.
> > >    # mount -t newnfs -o nfsv3 <any-server>:/path <mnt-path>
> > 
> > On 8-STABLE (both client and server). First test is NFSv3 on the
> > standard
> > client:
> > 
> > (0:842) new-gate:~terry# mount -t nfs -o nfsv4 new-rz1:/data /foo
> > [tcp6] new-rz1:/data: NFSPROC_NULL: RPC: Program/version mismatch; low
> > version = 2, high version = 3
> > [tcp] new-rz1:/data: NFSPROC_NULL: RPC: Program/version mismatch; low
> > version = 2, high version = 3
> > 
> > ^C
> > (1:843) new-gate:~terry# mount -t nfs -o nfsv3 new-rz1:/data /foo
> > [...]
> > (0:869) new-gate:/foo/Backups/Suzanne VAIO# dd if=0cff3d7b_VOL.spf
> > of=/dev/null bs=1m
> > 6010+1 records in
> > 6010+1 records out
> > 6301945344 bytes transferred in 69.730064 secs (90376302 bytes/sec)
> > 
> > Now, let's try the newnfs client (cache should have been primed by the
> 
> > first run, so we'd expect this to be faster):
> Just thought I'd mention that, since it is a different mount, the caches
> won't be primed, which is good, because that would mask differences.
> 
> > (0:879) new-gate:/tmp# umount /foo
> > (0:880) new-gate:/tmp# mount -t newnfs -o nfsv3 new-rz1:/data /foo
> > (0:881) new-gate:/tmp# cd /foo/Backups/Suzanne\ VAIO/
> > (0:882) new-gate:/foo/Backups/Suzanne VAIO# dd if=0cff3d7b_VOL.spf
> > of=/dev/null bs=1m
> > 6010+1 records in
> > 6010+1 records out
> > 6301945344 bytes transferred in 135.927222 secs (46362644 bytes/sec)
> > 
> > Hmmm. Half the performance. The problem isn't the disk speed on the
> 
> > server:
> Ok, good. You aren't seeing what the two guys reported (they were really
> slow, at less than 2Mbytes/sec). If you would like to, you could try the
> following, since the two clients use different default r/w sizes.
> 
> # mount -t newnfs -o nfsv3,rsize=32768,wsize=32768 new-rz1:/data /foo
> 
> and see how it changes the read rate. I don't know why there is a
> factor of 2 difference (if it isn't the different r/w size), but it
> will probably get resolved as I bring the experimental client up to date.
> 
> Thanks a lot for doing the test and giving me a data point, rick

root at jester1d / ->mount -t nfs -o wsize=65536,rsize=65536 
servant.ixsystems.com:/a/isos /mnt
root at jester1d / ->cd /mnt
root at jester1d /mnt ->dd if=PCBSD8-STABLE-20100420-x64-DVD.iso of=/dev/null 
bs=1m
3344+1 records in
3344+1 records out
3507386368 bytes transferred in 34.562502 secs (101479528 bytes/sec)

root at jester1d /mnt ->cd ..
root at jester1d / ->umount /mnt
root at jester1d / ->mount -t newnfs -o nfsv3,rsize=65536,wsize=65536 
servant.ixsystems.com:/a/isos /mnt
root at jester1d / ->cd /mnt
root at jester1d /mnt ->dd if=PCBSD8-STABLE-20100420-x64-DVD.iso of=/dev/null 
bs=1m
345+0 records in
345+0 records out
361758720 bytes transferred in 46.191718 secs (7831679 bytes/sec)

The first run hits network limits.

Both machines are nehalems, intel NICs, I can give details if needed.

-- 
Thanks,

Josh Paetzel
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part.
Url : http://lists.freebsd.org/pipermail/freebsd-fs/attachments/20100912/5f4990fc/attachment.pgp