copy 150G over 100Mbit

[Aleksey Kuznetsov]

On 2 Jun 2003 at 11:05, Michael Conlen wrote:

> 12.5MBytes/sec = 750 MBytes/min = 4500 MBytes/hour.

12,5MBytes/sec = 750 MBytes/min = 45000 Mbytes/hour 
                                                       ====

> 
> Two problems, protocol overhead and disk speed.
> 
> How fast can you read off that IDE system? Try tarring the filesystem
> and dumping it to /dev/null. If you can't get 12.5MBytes/sec then
> don't look to the network first.You should consider the size of your
> files. Sure your disks can move data fast, however if you have lots
> and lots of really small files your disks are going to be seeking all
> the time and reading very little of it. If your moving one large file
> there's less seek and more read. Also with lots of small files
> whatever protocol you use will have more overhead to eat more of your
> network bandwith. The difference can be an order of magnitude in disk
> throughput on reads.
> 
> Over the network you will end up with all kinds of overhead for
> various protocols. NFS isn't the fastest thing in the world. Rsync has
> it's uses, but I don't think this is one of them.
> 
> Consider using cpio over the network. Quick tests show that cpio gets
> better performance than tar by a long shot.
> 
> Another consideration might be to move the entire filesystem. If you
> create identical sized partitions on each system, instead of mounting
> them just copy the whole filesystem across using ssh or rsh. I haven't
> tried this on FreeBSD but it shouldn't be to hard to get working. The
> idea being that you read from /dev/ad0s1f on one box and write to
> /dev/ad0s1f on the other. Something like
> 
> dd if=/dev/ad0s1f | ssh hostname 'dd of=/dev/ad0s1f'
> 
> You can try playing with the bs parameter to get optimal read/write
> speeds. A quick test on my system shows that the disk throughput
> increases as the block size increases up to about 8k from
> 4.5MBytes/sec at 512 to 12MBytes/sec at 8k. On a RAID setup it's
> likely to be larger.
> 
> You may be able to dump to a larger partition on the destination
> system though the extra space would go unused. I don't have a good way
> to test this here.
> 
> --
> Michael Conlen
> 
> 
> Achim Patzner wrote:
> 
> >Am 02.06.2003 16:23 Uhr schrieb "Damir Horvat" unter
> ><damir at voljatel.si>:
> >  
> >
> >>I need to copy ~150Gbytes over switched 100Mbit network to new
> >>machine (in max 7-8 hours).
> >>
> >>Source machine (IDE RAID 5) is in production, iostat shows average
> >>transfer of 8-9Mbytes/sec. Destination box is FBC RAID 10.
> >>    
> >>
> >
> >Excuse me, but what would you expect using a 100 M*bit*/s network
> >connection?
> >
> >
> >Achim
> >
> >_______________________________________________
> >freebsd-performance at freebsd.org mailing list
> >http://lists.freebsd.org/mailman/listinfo/freebsd-performance
> >To unsubscribe, send any mail to
> >"freebsd-performance-unsubscribe at freebsd.org"
> >  
> >
> 
> 
> _______________________________________________
> freebsd-performance at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-performance
> To unsubscribe, send any mail to
> "freebsd-performance-unsubscribe at freebsd.org"