RFC - extra sync functionality...
Garance A Drosihn
drosih at rpi.edu
Fri Jun 13 13:24:28 PDT 2003
At 10:14 AM -0400 6/13/03, Ken Smith wrote:
>Taking that view of things Jun and others have already started
>to carve the FTP site into the equivalent of zones. The trick
>though is coming up with the equivalent of what triggers a Zone
>Transfer, which is a different serial number in the SOA for DNS.
Well, let me mention another partially-formed idea that I
have been thinking about...
Another issue is how to get a consistent collection of what you
are downloading, once you do start the download. I have been
hit by cases where I cvsup the source tree, and I get *part*
of some commit. Ie, when I go to do the buildworld, I get
errors because I managed to get *some* files from some commit,
but not *all* of the files that were changed by that single
commit. From this comes the common advice "Just CVSUP again
and the problem should go away".
Let's say my machine does decide to start downloading. At the
very time I'm downloading from some server, that server might
be downloading from *its* source. Neither download can be an
atomic operation, so the end user can end up with an inconsistent
collection of files.
It occurred to me that we could solve this by using snapshots
on the mirrors (by this I mean the UFS-snapshots which are
related to softupdates, and available in freebsd-5.x). When I
connect to a mirror, I would not be downloading from "the real
disk", but the mirror would instead connect me to a snapshot of
the real disk. The mirror would only create a new snapshot when
it was finished updating from it's source, so when any given
snapshot is made the repository should be in a completely
Assuming that all works, the question was what event would the
super-master source site use for creating it's snapshot. I
was thinking it could create a snapshot, and then start the
buildworld-testing processes. If the buildworld succeeds, then
it would make *that* snapshot the one used for any subsequent
connections to the server. (I am still a bit undecided about
how all that should work, however...) I did start playing with
some experiments along this line last year, but was tripped up
by some bugs that were still in the UFS-snapshot code at that
The relevance of this to your comments is that you would end
up with a single "serial number" for an entire snapshot, and
that would be *in* the snapshot. So, you don't need serial
numbers for every file, you can have a single number for the
entire repository. (or maybe break the single repository into
subsets, and then have a serial number per subset).
I realize this is another idea which would need a bit more
fleshing-out before it would be practical to use, but I also
think that it's a promising idea.
Garance Alistair Drosehn = gad at gilead.netel.rpi.edu
Senior Systems Programmer or gad at freebsd.org
Rensselaer Polytechnic Institute or drosih at rpi.edu
More information about the freebsd-hubs