Subversion/CVS experiment summary

[Craig Boston]

On Monday 09 February 2004 11:53 am, Stijn Hoop wrote:
> Did you have to modify the script, or pass unusual options? I'd like to
> reproduce this, but I didn't get very far when I tried a few days ago with
> the 0.37.0 version of the tool.

No, I used the script as-is.  The version I have is LastChangedRevision: 8527, 
with a date of Jan. 29.  It looks like that version is slightly newer than 
the one included with 0.37 (Rev 8512).

One thing that may have made a difference is that so far I've been importing 
things in chunks rather than trying to do the whole repo at once.

> but although it looks like it handles things much better (even vendor
> branches etc), it loads EVERYTHING into memory -- which means that it
> eventually grew to 1G of memory/swap at which point my memory was
> exhausted, and this was at pass 2 of 7...

Does the Python version do the same thing?  I didn't think to look at memory 
usage very closely while it was running :-/

> > Comments on importing: It's SLOOOOOOOOOOW.  It took 43.9 hours just for
> > src/sys, and this is a relatively speedy system!  It starts out at a
> > pretty good pace, but the more commits it processes, the slower each one
> > seems to take.
>
> That doesn't bode well. Is committing in the new SVN repository also slow?

No, creating a new branch and committing new and changed files to it seems to 
be just as quick as with an empty repository.  I haven't delved into the 
script enough to know for sure, so this is a wild guess, but I think the 
speed issue has to do with the script itself.  I'm guessing that the method 
it uses to track the status/branch/etc. of the RCS files is subject to a 
linear slowdown.

I'm going to attempt to verify this by doing a dump / load cycle on the repo 
that everything was imported into.  If it goes quickly then we can assume 
it's the conversion script.  If not, then there are bigger problems...

> My thoughts were now going to do something like Tom Lord proposed for an
> Arch gateway -- just import a CVS working copy into SVN at a certain
> cut-off date, and setup a bi-directional gateway between the two.

If I'm reading that right, it sounds similar to a thought I had about just 
routinely checking out snapshots and committing them on a vendor branch.  Of 
course you'd have to do that separately for each branch you're interested in.  
IMO, I find it immensely useful to have the entire history of a file at hand.

> Actually, would a sort of access control wrapper that is now also used with
> the FreeBSD CVS repository not work? I do agree that it would be nice to
> have per-directory access control with the svnserve method.

Yes, I think the same sort of access hooks (pre-commit?) can be used.  The 
Subversion manual even mentions that, I just forgot about it...

That method has always seemed a little... hackish to me.  You still need write 
access (file permissions) to almost the entire repo so it does nothing 
against editing the files directly -- though with SVN this is a little more 
difficult as it's all bdb files rather than plain text.  Maybe there's a more 
secure way to do it with a restricted shell that I just don't know about.

> > [ repeated merges ]
> Which is a shame, this would be a major selling point. On the other hand,
> considering the amount of work done and the fact that it really works quite
> well already (at least for my small repository) should make people want to
> switch :)

In the release notes for 0.37 there is a brief blurb about "'svn merge' now 
notices ancestry by default".  I'm not sure exactly what that means or if 
it's related...

> > It also looks as if Subversion 0.37 (aka 1.0-RC) has just been released.
> > I'll have to take a look at it and see if any of the problems noted above
> > have been resolved.
>
> Please let me know the results!

Will do!  My local Subversion server (one running Apache, not the machine I've 
been doing the tests on) had just finished upgrading the port.  I don't think 
it needs a dump/load cycle but I'm doing one anyway just to be safe...

Craig