SUMMARY: Fast releases demand binary updates..
rwatson at FreeBSD.org
Sat Jan 14 07:16:13 PST 2006
On Thu, 12 Jan 2006, Jo Rhett wrote:
<much unhappiness elided>
> For now we'll have to see if Colin can find ways around the problems without
> the tools he needs to do it right.
It seems fairly clear to me that Colin's tools provide useful building blocks
that can be used to create something like what you're looking for: it is an
efficient way to generate, manage, and distribute difference sets between
snapshots of a system. What Colin's pieces explicitly don't provide is a
administrative infrastructure for imposing heavy weight packaging semantics
and version information -- specifically, an explicit version database,
roll-back, and a mechanism for configuration merges. I'm sure many sites
would like to have the heavier weight mechanism, but it's certainly a
non-trivial project that will take a lot of time for someone to get right.
Someone(tm) will need to spend several months on it to get it really working
well, but I think it would be well worth the effort. I do suspect, however,
that the starting point for this is to discard the formal project releases,
and instead simply provide infrastructure for someone with their own notion of
what releases are, and what versions make sense. That way if a site chooses
to deploy our stock releases or patches, that's fine, but they can also manage
their own custom snapshots.
Presumably a first starting point is to take the assumption that the build
system will produce a series of install snapshots, presumably using
installworld/installkernel to a specific DESTDIR, and that all updates will be
generated between them, with version names and relevant difference
combinations to be assigned to them by the admin.
The second part will be handling what's normally done by mergemaster in some
way -- for some environments, simply generating a second tarball that contains
the output of distrib-dirs (etc) and then providing that to bootstrap
mergemaster rather than a source tree would be sufficient. Other environments
might wish to simply splat down replacement files based on having run
mergemaster in advance on the build machine. A bit of minimal but
well-crafted glue is presumably required here, such as the ability to point
mergemaster at the update blob and tell it "Yeah, so, I want my / to be merged
from here, not /usr/src/etc".
The final part is the meta-data for these snapshots, both carried with them,
and on the target system. If using Colin's binary difference pieces, they
will often be quite compact, and it's easy to imagine storing the necessary
information to roll back or forwards by several revisions in a relatively
small amount of space. Tricky things include how to accomplish relative
atomicity in updates. If we assume that the snapshot itself, along with
possibly a version name/number (from, to), is basically sufficient meta-data,
that's quite advantageous. It becomes the responsibility of the admin to not
screw up by assigning the same name to multiple versions, etc.
But as always, the tricky thing is someone actually doing the work. It's a
non-trivial and time-consuming task, and isn't just a weekend's hacking.
There will almost certainly be a number of "Oh damn, I didn't think of
that!"'s along the way. I don't think anyone is opposed to it happening, but
getting someone to do all that work with little recompense (or even
significant recompense) requires some amount of convincing!
Robert N M Watson
More information about the freebsd-stable