mercurial working copy of FreeBSD src

[Giorgos Keramidas]

On Fri, 8 Aug 2008 11:47:15 -0700, "Navdeep Parhar" <nparhar at gmail.com> wrote:
> Hello everyone,
>
> I'm looking for the fastest way to get a full mercurial repository of
> HEAD.

Do you really want the *FULL* history of head?  It's probably going to
be in the order of a couple of hundred of MB, or even more.

If you can live with an Hg repository that only includes the 2008
history of 'HEAD', then please keep reading, you may like the rest of
this post :)

> I tried using hgsvn and the tree at http://svn.freebsd.org/base/head
> but it looks like the first hgpullsvn operation will take days (literally):

hgpullsvn is slow.  Really really *REALLY* slow...  It's probably a
better idea to get a recent version of the mercurial/crew branch.  This
includes support for the svn-1.5.X bindings, and you can convert parts
of the history of the /base/head branch.

I have a local hg tree with the history of the 'head' branch since
2008-01-01 on my laptop, which I periodically resync from svn with:

    keramida at kobe:/home/keramida$ cd /hg/bsd
    keramida at kobe:/hg/bsd$ ./pull-head.sh
    scanning source...
    sorting...
    converting...
    [lots of output snipped]
    keramida at kobe:/hg/bsd/head$ hg shortlog --limit 2
    4201:99f1acdec12f | 2008-08-09 04:08 +0000 | imp: Rather than waiting a fixed amount of time, which might not be enough
    4200:d5553c08bf90 | 2008-08-09 03:54 +0000 | imp: Change -1 to 0xfffffffful since the interface returns uint32_t.



(1) A bit of background
=======================

I didn't really want to convert the full history of the 'head' branch
From Subversion, because I have a local Subversion repository for that.
Being able to rebase my local in-progress patches on an Hg clone _did_
seem useful at the time, so that's why I converted only the history of
'head' from the start of 2008.

The initial pull was done by pulling a clean copy of the first commit of
2008 and then importing changesets on top of that with:

    % cd /hg/bsd
    % \time hg convert --config convert.svn.startrev=175026 \
          --config convert.svn.trunk='head' \
          --config convert.svn.branches='' \
          --config convert.svn.tags='' file:///home/svn/base/ head

The initial pull took a bit of time, but far less than hgimportsvn.
Subsequent pull-head.sh runs take literally seconds to pull from a local
Subversion repository mirror.  The speed is much much better than
hgpullsvn, because the Hg convert extension doesn't parse the output of
shell commands, but it links `natively' (through SWIG) with the
subversion libraries.

The conversion process is interruptible, and if you keep the 'SHA map'
of the initial conversion you can keep pulling changesets into the
conversion clone as many times as necessary with something like:

    % cd /hg/bsd
    % cat -n pull-head.sh
         1  #!/bin/sh
         2
         3  hg convert \
         4    --config convert.svn.trunk='head' \
         5    --config convert.svn.branches='' \
         6    --config convert.svn.tags='' \
         7    file:///home/svn/base/ head
    %

You don't need the `convert.svn.startrev=175026' option in the second
and subsequent runs, because the conversion clone already has a file
called `.hg/shamap' and the conversion picks up from the place it
stopped the previous time.



(2) Bootstrapping your own clone
================================

Everything above is of `encyclopedical' nature, because now that I've
done the conversion once, you can just use a bundle file and a matching
copy of my `.hg/shamap' file to bootstrap a conversion clone of your
own.

You can fetch an Hg bundle and a matching shamap with the history of
'head' in 2008 from:

    http://people.freebsd.org/~keramida/mercurial/head.hg
    http://people.freebsd.org/~keramida/mercurial/head.shamap

The checksums of these files should be:

    MD5 (head.hg) = 4978723c560bc48c390be1634c7d36a4
    MD5 (head.shamap) = 8e5957657fb499ed3a3575fc921e3029
    SHA256 (head.hg) = ab073135925fdb54a2820077a5a4d75f4306d44a217b7c30d6b8719a9a5008d4
    SHA256 (head.shamap) = 04c6648bd108d3550ffb126c321b8faf2f5dda9f891787bb57ae178efa271893

The head.hg bundle is around 92 MB and you can pull it directly into an
empty Hg workspace:

    % hg init head
    % cd head ; hg pull /var/tmp/head.hg

Then copy the `head.shamap' file in `.hg/shamap' and you are ready to
go.  You can start cloning this tree, and you can also run "hg convert"
to incrementally pull the history of the svn/head branch, as many times
as necessary.  The `pull-head.sh' script I included above will make
subsequent pulls less tedious by saving you the typing of all the
--config options to get the svn->hg branch names right.



(3) Important disclaimer
========================

Note that this is *NOT* an official branch of the FreeBSD source tree,
so if you plan to use the head.hg bundle, this email is _not_ really a
promise to keep uploading `bootstrap bundles'.  You are also the sole
person responsible for comparing the source tree checked out from the Hg
clone with a subversion or cvs checkout and making sure that it matches.

So have fun if you like it, but please don't sue me or something.

HTH,
Giorgos

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 194 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20080809/951d3614/attachment.pgp