Strange performance characteristics with ZFS

[Steven Schlansker]

Brian Donnell wrote:
> That was my fault, I misread what you had said your configuration was. 
> The problem I ran into was the same as you're seeing with the system
> waiting on ZFS causing the NFS client to hang and everything becoming
> unresponsive.  The ls script should be running on the ZFS machine and
> executing ls on the ZFS directory.  If you have a smaller file to try
> with first I'd recommend it.  30GB of corrupt data is just pain.  To be
> honest, I gave up on NFS with ZFS and started using Samba for everything
> after compiling samba3 without a couple functions (check the ZFS vs
> Samba debugging results thread from earlier this month) as it actually
> maxes out a 100Mbit network link for the transfer.  That's something I
> could never get NFS with ZFS to come close to doing.
> 
> Just to be sure, since you're using nfsd, that means you have the
> sharenfs option of the zfs pool turned off?
> 
> -- Brian
> 
> 

No, I have it set up like this:
[steven at universe ~]$ sudo zfs get sharenfs
NAME                      PROPERTY  VALUE                     SOURCE
universe                  sharenfs  -mapall=steven            local
universe/backup           sharenfs  -mapall=steven            inherited
from universe
universe/backup/engineer  sharenfs  -mapall=steven            inherited
from universe
universe/backup/zoo       sharenfs  -mapall=steven            inherited
from universe
universe/data             sharenfs  -mapall=steven            inherited
from universe
universe/incoming         sharenfs  -mapall=steven            inherited
from universe
universe/kabamba          sharenfs  -mapall=steven            inherited
from universe
universe/media            sharenfs  -mapall=steven            inherited
from universe
universe/music            sharenfs  -mapall=steven            inherited
from universe
universe/platypus         sharenfs  -maproot=root             local


So here's the score:

Write to ZFS on local machine (dd if=/dev/zero of=file) = 36.2 MB/s
This still seems a bit slow, but it's not bad so I'll let it slide :-p

dd/netcat across network = 24.8 MB/s
Also seems a bit slow, considering it's gigabit.

Write to ZFS over nfs (dd again) = 16MB/s
It does the whole ping-ponging speed (goes at like 25MB/s for a little
bit, drops to 0, back to 25, etc)

Any more ideas?  Thanks!
Steven