What would be the best way to copy lots of files from oneserver to another.

[Sean J. Schluntz]

On Saturday, June 21, 2003, at 01:08  PM, Tom Samplonius wrote:

>
> On Sat, 21 Jun 2003, Simon wrote:
>
>> I recently used rsync to copy ~35gb worth of data, 1,170,168 inodes,
>> without any problem over 100mbps interface. Took 3 or so hours.
>> You do need lots of RAM, if you swap, you're screwed. RAM
>> usage would depend on number of files you need to copy at once.
>>
>> -Simon
>
>   Yes, rsync uses large amounts of memory when the fileset is large.
> Plus, depending on your CPU, you might want to avoid using ssh 
> encryption
> too.
>
>   Usually if it is a LAN to LAN copy, cpdup over NFS will be WAY faster
> than rsync, and use little memory.  I've used cpdup for moving mail to 
> a
> new server.  I synced before the cutover, shutdown the mail server, 
> synced
> one more, and then brought up the mail server on the system.  Very 
> tidy.
>

even faster than that:

cd sourcedir ; find . -print | cpio -o | \
ssh user at host "cd destdir ; cpio -pdmuv"

Drop the v to keep from cluttering your screen. You might need to add c 
to both ends for 'compatibility' if it's different arch or vers of 
cpio, but I have better luck with it.

I have yet to find anything faster over long links, use rsh for local 
and it's even faster.

-Sean