cannot witness rsync delta-algorithm

Christopher J. Ruwe cjr at cruwe.de
Thu Mar 13 16:55:36 UTC 2014


Having read the rsync delta transfer paper included with the rsync
documentation (Andrew Tridgell, Paul Mackerras: The Rsync Algorithm),
I wanted to witness this in practice. 

(rsync should use rolling MD5 checksums on source and target file chunks
to determine if the data is present on remote target without having to
transfer the data.)

I do not succeed. I have setup a file 'here' using dd and /dev/zero:

[cjr at dijkstra:~/rsync-exp]$ dd if=/dev/zero of=here bs=1M count=100
100+0 records in
100+0 records out
104857600 bytes transferred in 0.101729 secs (1030755108 bytes/sec)

I used 100 MB to ensure that changing delta-window-size would not be
an issue.

I then used rsync to copy 'here' to 'there'. Afterwards, I changed the
last MB of here:

[cjr at dijkstra:~/rsync-exp]$ rsync -vh --inplace here there
sent 104.88M bytes  received 35 bytes  69.92M bytes/sec
total size is 104.86M  speedup is 1.00
[cjr at dijkstra-old:cb-krb/rsync-exp]$ diff here
there                                       

here and there obviously have equal content.

Then, I changed the last 1 MB of here:

[cjr at dijkstra:~/rsync-exp]$ dd if=/dev/random of=here bs=1M count=1
seek=99 1+0 records in
1+0 records out
1048576 bytes transferred in 0.026790 secs (39140716 bytes/sec)

'here' and 'there' differ:

[cjr at dijkstra:~/rsync-exp]$ diff here there
Files here and there differ


From what I understand from rsync workings, I would expect another
rsync from 'here' to 'there' only to copy some MBs due to possibly
different window sizes, but significantly less than 100MB. Yet, it
does not. Why?

[cjr at dijkstra:~/rsync-exp]$ rsync -vhc --inplace here there
sent 104.88M bytes  received 35 bytes  69.92M bytes/sec
total size is 104.86M  speedup is 1.00
[cjr at dijkstra:~/rsync-exp]$ diff here there   

'here' has been copied as a whole to 'there'. The '-W' flag has not
been set!

My rsync has been compiled thus:

[cjr at dijkstra:~]$ pkg info rsync
rsync-3.1.0_1
Name           : rsync
Version        : 3.1.0_1
Installed on   : Mon Mar 10 21:23:56 CET 2014
Origin         : net/rsync
Architecture   : freebsd:10:x86:64
Prefix         : /usr/local
Categories     : net ipv6
Licenses       : GPLv3
Maintainer     : ehaupt at FreeBSD.org
WWW            : http://rsync.samba.org/
Comment        : Network file distribution/synchronization utility
Options        :
        ACL            : off
        ATIMES         : off
        DOCS           : on
        FLAGS          : off
        ICONV          : off
        POPT_PORT      : off
        RENAMED        : off
        RENFL          : on
        SSH            : on
        TIMELIMIT      : off
        ZLIB_BASE      : on
[...]

Can anybody explain this behaviour?


Thanks and cheers,
-- 
Christopher

TZ:         GMT + 1h
GnuPG/GPG:  0xE8DE2C14
 
FreeBSD 10.0-STABLE #0 r262291+cf0fa48(stable/10): Sat Feb 22 02:01:26
CET 2014 cjr at dijkstra.hb22.cruwe.de:/usr/obj/usr/src/sys/GENERIC 
 
Punctuation matters:
"Lets eat Grandma or Lets eat, Grandma" - Punctuation saves lives.
"A panda eats shoots and leaves" or "A panda eats, shoots, and leaves" -
Punctuation teaches proper biology.

"With sufficient thrust, pigs fly just fine. However, this is not
necessarily a good idea. It is hard to be sure where they are going to
land, and it could be dangerous sitting under them as they fly
overhead." (RFC 1925)


More information about the freebsd-questions mailing list