zfs -> ufs rsync: livelock in wdrain state

[Dmitry Morozovsky]

On Tue, 8 Jan 2013, Konstantin Belousov wrote:

> > Now, during last rsync, the process is stuck as

[snip]

> > root at moose:/ar# sync
> > load: 0.00  cmd: sync 67229 [wdrain] 468.17r 0.00u 0.00s 0% 596k
> > 
> > Any hints? Quick searching throug freebsd mailing lists and/or open PRs does 
> > not reveal much.
> > 
> 
> Are there any kernel messages about the disk system ?
> 
> The wdrain means that the amount of the dirty buffers accumulated exceeds
> the allowed maximum. The transient 'wdrain' state is normal on a machine
> doing lot of writes to a filesystem using buffer cache, say UFS. Failure
> to clean the dirty buffers is usually related to the disk i/o stalling.
> 
> It cannot be denied that a bug could cause stuck 'wdrain' state, but
> in the last five or so years all the cases I investigated were due to
> disks.

Yes, it seems so:

root at moose:~# camcontrol devlist
load: 0.03  cmd: camcontrol 49735 [devfs] 2.68r 0.00u 0.00s 0% 820k

and then machine is in well known "hardly alive" state: TCP connects 
established, process switching does not go.

Will investigate the hardware, thank you.

-- 
Sincerely,
D.Marck                                     [DM5020, MCK-RIPE, DM3-RIPN]
[ FreeBSD committer:                                 marck at FreeBSD.org ]
------------------------------------------------------------------------
*** Dmitry Morozovsky --- D.Marck --- Wild Woozle --- marck at rinet.ru ***
------------------------------------------------------------------------