svn - but smaller?

[Isaac (.ike) Levy]

One of my teammates and I were just doing a write up on this very issue,

On Jan 23, 2013, at 10:55 AM, Mike Tancsa wrote:
> On 1/23/2013 10:37 AM, Oliver Brandmueller wrote:
>> 
>> But my main concern is the system sources anyway. freebsd-update is not 
>> feasible for me, as described in the original post.
>> 
> Actually, if you build the port minus the NEON option, its as bad in
> terms of dependencies.

--------
THE UGLY

Source for SVN:         496M (+)
Source for FreeBSD 9.1: 746M (actual)

(df output details below)

-------
THE BAD

After 15+ years of FreeBSD use, I remember what a great thing cvsup was when it hit.
However, SVN presents several problems for OS use (again):

1) License.  Many of SVN's dependencies will never be available in the FreeBSD source.
While this is totally OK for development, SVN is 3rd party software, this is unacceptable to force as 'the' respected path for OS source builds.

2) Heft: cvsup/csup was excellent for 1 thing: grabbing a REL branch.  Perhaps grabbing STABLE or CURRENT.  Systems administrators could QA/test new branches on massive numbers of servers quickly and efficiently.

--------
THE GOOD

We've just resolved this for ourselves, and are wrapping it in a clean sh script:
(I'd love to know where we can send it for input when we're done?)

40 lines of shell could get the jist of what users really need:

- Download the src using fetch(1)
  fetch ftp://ftp.freebsd.org/pub/FreeBSD/releases/amd64/9.1-RELEASE/src.txz
- Extract the tarball (to /usr/src, or wherever).
  tar -xf src.txz

We're hacking out some simple script add-ons out right now for ourselves, to make this more CVSUP like:
+ flag to keep source tarball somewhere (instead of merely un-tarring it in a pipe)
+ flag to un-tar source to a particular directory
+ flag to specify OS version
+ flag to specify RELEASE, STABLE, CURRENT
  (if they exist, CURRENT may be tricky?)
+ define source server/mirror

+ source config in /etc, (or rc.conf ?), if exists

--
Nice-to-have extra features (some necessary for us):
+ define protocol (tricky), e.g. ftp/http/https/other
+ after unpacking, run against mtree (possibly kept on separate server or locally) to validate sources
  + exit non-zero if particular conditions exist
+ checksum tarball (possibly against checksums kept on separate server or locally) to validate sources
  + exit non-zero if particular conditions exist
+ flags to override/change tar options

++ also nice to have, more cvsup features, (I need to read through man page again for a sanity check)


--
Regarding SVN:

I know the SVN change is a profound leap foreword in source management and collaboration, (I've carried many shops through CVS/SVN/GIT migrations as an SA).

Developing/hacking in the FreeBSD source is already simpler, though as an outsider, (no commit bit), the transition has been expectedly rough-edged :)

On Jan 23, 2013, at 10:06 AM, Ronald Klop wrote:
> I've read about this initiative.
> http://svnweb.freebsd.org/base/user/des/svnsup/
> Maybe you can help there.

Aside from the heft/licence issues I noted above, it's a bit late to consider this, cvsup is going away:

- the *ports* CVS/csup infrastructure is going to be disabled on Feburary 28th

- the *source* CVS/csup infrastructure is deprecated, but doesn't have a definite end-date


On Wed, Jan 23, 2013 at 04:12:22PM +0100, Frank Staals wrote:
>> I'm kind of surprised for the need of this though. Why not simply use
>> portsnap if you are not actively developing ports? 
> 
On Jan 23, 2013, at 10:37 AM, Oliver Brandmueller wrote:
> But my main concern is the system sources anyway. freebsd-update is not 
> feasible for me, as described in the original post.


For users/administrators, to merely fetch OS sources for a given branch, it goes against the grain of nearly every reason users use FreeBSD to say 'just use svn'.

Additionally, it's not been fun recently to 'just use portssnap', when the actual binary ports servers have gone through the recent security incident, (as well as all the changes).

I'm not meaning to be negative here, but this has slid pretty far away from the ideals that *BSD users care about.

Best,
.ike






# uncompressed canonical sources for svn (I believe I missed some dependencies of the dependencies)
du -d 1 -h
5.5M	./apr-1.4.6
5.3M	./sqlite-amalgamation-3071300
 13M	./libtool-2.4.2
 79M	./perl-5.16.2
4.0M	./neon-0.29.6
 66M	./Python-2.7.1
153M	./db-5.3.21
 55M	./subversion-1.7.8
8.6M	./m4-1.4.16
3.0M	./expat-2.1.0
 12M	./pkg-config-0.27.1
3.0M	./gdbm-1.10
 67M	./gettext-0.18.1.1
 21M	./libiconv-1.14
496M	.

## FreeBSD 9.1 Source
$ pwd ; du -d 1 -h
/usr/src
3.2M	./bin
 11M	./cddl
316M	./contrib
 40M	./crypto
2.0M	./etc
3.7M	./games
5.9M	./gnu
1.1M	./include
484k	./kerberos5
 31M	./lib
2.1M	./libexec
1.3M	./release
 32k	./rescue
7.2M	./sbin
3.6M	./secure
 39M	./share
200M	./sys
 44M	./tools
 13M	./usr.bin
 18M	./usr.sbin
746M	.