process size

[Jason Evans]

Alexandre Biancalana wrote:
>> You're running your system as i386 rather than amd64, right?  It looks
>> like rsync is exhausting its address space while trying to reallocate
>> the growing (and apparently very large) file list.  There's nothing
>> surprising here to me.  If you use amd64 rather than i386 you won't have
>> this problem, though you will still see poor performance due to
>> swapping.  The most prudent solution is probably to use multiple rsync
>> calls to copy portions of your data at a time.
> 
> I agree that this could be the problem... but I´m not running i386.
> 
> # uname -a
> FreeBSD  7.0-CURRENT FreeBSD 7.0-CURRENT #2: Fri Sep 21 18:06:33 BRT
> 2007     root@:/usr/obj/usr/src/sys/BACKUP  amd64

Hmm, okay, so much for that theory.  I blame rsync.  If you look it its 
util.c, you will see that it can't allocate over 2^31 bytes per 
allocation (MALLOC_MAX).  Additionally, the code uses (unsigned int) in 
many places that would have to be changed to (unsigned long) or (size_t) 
in order for things to work correctly with a larger MALLOC_MAX. 
Basically, rsync loses.

Jason