long, large rsync slows down over time ?

[Alex Zbyslaw]

Ensel Sharon wrote:

>I am moving a large number of files (about 40 GB, comprising almost 3.5
>million files) from one machine to another using rsync.
>
>I am noticing that the longer this rsync process runs, the slower it seems
>to go, and the more CPU time it is taking on the destination (FreeBSD
>6.1).
>
>Is there a reason for this ?  Is there some aspect of rsync (or
>FreeBSD) that just starts to bog down after transferring 1-2 million
>files?
>
>When the job started, load on the destination was between .2 and .8, and
>now, four days later, it is up to 4.0+  ...  and as I say, although I have
>not measured, throughput does seem to be slowing down.
>  
>
Are you using rsync with ssh?  If so try openssh-portable port with hpn 
options enabled.  If the machines are local you can even turn off ssh 
data encryption with -z which will make rsync go faster.

If machines are local you could also try using rsh, or take the risk for 
remote machines.

Though if it's been running for days, they don't sound local :-)

--Alex