Migrate large amount of small files
Ckcheng
net.help at m2k.com.tw
Sat Oct 10 16:23:24 UTC 2009
Hi all,
Currently, I have a directory with over 5M small files (1~32K). Now,
I want to transfer this directory to another machine and found
that it's extremely slow and painful process. I tried the following
method:
1. rsync
2. tar via ssh
3. tar via nc
(all take hours and hours to finish)
None of them is able to give me a reasonable migration time. So, I'm
here for asking help. Any suggestion is extremely welcomed. Thank you.
Btw, here is brief information of my server. (both machines are the same)
OS: FreeBSD 6.4-Stable 64Bit
CPU: 2 x Xeon L5420 2.50GHz
RAM: 2 x 2G ECC DDR2-667 (full buffered)
DISK: Seagate Barracuda ES 16MB (SATA 300)
Network: 1Gbps (Broadcom BCM5708)
Filesystem: UFS2
Regards,
More information about the freebsd-performance
mailing list