A completely strange ZFS hang.

[Zaphod Beeblebrox]

I recently copied my entire ZFS pool off into a temporary ZFS pool
with "zfs send -R".  Marvelous way to do things.

Anyways... I reformatted my ZFS pool (more RAIDZ1 disks) and started a
copy back... something along the lines of:

zfs send -R zroot/vr2 | ssh otherhost "zfs receive -Fd vr2"

... turns out this puts the stuff from zroot/vr2 into vr2/vr2 --- but
I can rename it later.

While this is ongoing (5T of data at about 2T per day transfer ---
probably could go faster, but I havn't tried), I needed to download
some stuff over my 50 megabit link.  So sitting in an NFS mounted
directory (mounted -3 -T), I:

mv somedir1 /vr2/new/

Where /vr2/new is a new ZFS filesystem not related to the heirachy
that the "zfs receive" is creating in vr2/vr2.

At some point I realize I need something before I need somedir1, so I
CTRL-Z that command and:

mv somedir2 /vr2/new/

... sometime after this command starts, ZFS locks hard.  zfs list or
even an ls -l inside the zfs dir is locked hard.  The zfs receive
process has gone idle too.

Because I don't want to restart everything, I try backgrounding the
first mv.  Surprisingly, this works ?!?

Keep in mind while thinking about this problem that the locked
condition occurred for about 20 minutes while I tried to work it out.
The machine running ZFS was not locked (it has a UFS root and standard
commands were working fine)