zfs -> ufs rsync: livelock in wdrain state

[Dmitry Morozovsky]

Dear colleagues,

I have archive server with pretty large ZFS (24*2T in single raidz2 raidgroup)

Sometimes we moved really old archives to external SATA drives, which are 
formatted with UFS2/SU.  Files are copied via rsync

The system in question is stable/8; upgrade to stable/9 is planned, but not yet 
completed.

Now, during last rsync, the process is stuck as

dump.2012062219.bin.gz
  3208015437 100%  102.42MB/s    0:00:29 (xfer#66, to-check=196/721)
dump.2012062220.bin.gz
load: 0.01  cmd: rsync 47543 [wdrain] 1904.69r 443.01u 241.12s 0% 1736k
^C
rsync error: received SIGINT, SIGTERM, or SIGHUP (code 20) at rsync.c(645) 
[sender=3.0.9]

As we can see, rsync writer stops in wdrain state.

I terminated it by ^C in terminal session, as it was not autogenerated 
backup.

Now, zfs and other system is working seemingly well, but trying to sync 
manually stucks console forever:

root at moose:/ar# sync
load: 0.00  cmd: sync 67229 [wdrain] 468.17r 0.00u 0.00s 0% 596k

Any hints? Quick searching throug freebsd mailing lists and/or open PRs does 
not reveal much.

Thanks!


-- 
Sincerely,
D.Marck                                     [DM5020, MCK-RIPE, DM3-RIPN]
[ FreeBSD committer:                                 marck at FreeBSD.org ]
------------------------------------------------------------------------
*** Dmitry Morozovsky --- D.Marck --- Wild Woozle --- marck at rinet.ru ***
------------------------------------------------------------------------