copy 150G over 100Mbit
meconlen at obfuscated.net
Mon Jun 2 08:05:26 PDT 2003
12.5MBytes/sec = 750 MBytes/min = 4500 MBytes/hour.
Two problems, protocol overhead and disk speed.
How fast can you read off that IDE system? Try tarring the filesystem
and dumping it to /dev/null. If you can't get 12.5MBytes/sec then don't
look to the network first.You should consider the size of your files.
Sure your disks can move data fast, however if you have lots and lots of
really small files your disks are going to be seeking all the time and
reading very little of it. If your moving one large file there's less
seek and more read. Also with lots of small files whatever protocol you
use will have more overhead to eat more of your network bandwith. The
difference can be an order of magnitude in disk throughput on reads.
Over the network you will end up with all kinds of overhead for various
protocols. NFS isn't the fastest thing in the world. Rsync has it's
uses, but I don't think this is one of them.
Consider using cpio over the network. Quick tests show that cpio gets
better performance than tar by a long shot.
Another consideration might be to move the entire filesystem. If you
create identical sized partitions on each system, instead of mounting
them just copy the whole filesystem across using ssh or rsh. I haven't
tried this on FreeBSD but it shouldn't be to hard to get working. The
idea being that you read from /dev/ad0s1f on one box and write to
/dev/ad0s1f on the other. Something like
dd if=/dev/ad0s1f | ssh hostname 'dd of=/dev/ad0s1f'
You can try playing with the bs parameter to get optimal read/write
speeds. A quick test on my system shows that the disk throughput
increases as the block size increases up to about 8k from 4.5MBytes/sec
at 512 to 12MBytes/sec at 8k. On a RAID setup it's likely to be larger.
You may be able to dump to a larger partition on the destination system
though the extra space would go unused. I don't have a good way to test
Achim Patzner wrote:
>Am 02.06.2003 16:23 Uhr schrieb "Damir Horvat" unter <damir at voljatel.si>:
>>I need to copy ~150Gbytes over switched 100Mbit network to new machine (in max
>>Source machine (IDE RAID 5) is in production, iostat shows average transfer of
>>8-9Mbytes/sec. Destination box is FBC RAID 10.
>Excuse me, but what would you expect using a 100 M*bit*/s network
>freebsd-performance at freebsd.org mailing list
>To unsubscribe, send any mail to "freebsd-performance-unsubscribe at freebsd.org"
More information about the freebsd-performance