Reducing the size of the ports tree (brainstorm v2)

[Alex Kozlov]

On Sat, Nov 01, 2014 at 12:07:23AM +0100, Tijl Coosemans wrote:
> On Fri, 31 Oct 2014 19:56:21 +0100 Baptiste Daroussin <bapt at FreeBSD.org> wrote:
> > tijl@ spotted an interesting point, distinfo and pkg-descr files files
> > convenient are taking a lot of space for "free", we can reduce the size of the
> > while ports tree by a factor 2 by simply merging them into one of the other
> > files (Makefile and/or pkg-plist) from my testing it really devides
> > significantly the size of the tree.
> > 
> > Problem is how to merge them if we want to.
> > 
> > What we do not want to loose:
> > - Easyness of parsing distinfo
> > - Easyness to get informations about the description
> > 
> > so far I have not been able to figure out a user friendly way
> > 
> > Ideas I got so far only concerns pkg-descr:
> > Adding an entry in the Makefile for the WWW:
> > WWW= bla
> > or an entry in the plist: @www http...
> > 
> > for the description the Makefile is not suitable as multi line entry in
> > Makefiles are painful
> > Maybe a new keyword:
> > @descr <<EOD
> > mydesc
> > in 
> > multiline
> > EOD
> > 
> > which could easily be added to the plist parser in pkg. But I'm do not find that
> > very friendly in particular for make(1) to extract the data.
> > 
> > Concerning the distinfo I have no idea.
> > 
> > so this mail is a call of ideas :), if nothing nice ideas is found we will just
> > do nothing here :)
> 
> For distinfo I was thinking about something like this in the Makefile:
> 
> DIST_FILES=	FOO BAR
> 
> FOO_FILE=	foo-1.0
> FOO_SITES=	http://www.example.com/foo/
> FOO_SIZE=	12345
> FOO_SHA256=	0123456789abcdef.....
> 
> BAR_FILE=	bar-2.0
> BAR_SITES=	http://www.example.com/bar/
> BAR_SIZE=	54321
> BAR_SHA256=	.....fedcba9876543210
> 
> If bsd.port.mk then contained this:
> DIST_FILES?=	DIST
> DIST_FILE?=	${PORTNAME}-${DISTVERSIONPREFIX}${DISTVERSION}${DISTVERSIONSUFFIX}
> 
> Then for ports with a single distfile named after PORTNAME (the large
> majority), the above would reduce to:
> 
> DIST_SITES=	http://www.example.com/foo/
> DIST_SIZE=	12345
> DIST_SHA256=	0123456789abcdef.....
> 
> Also, if BAR_SITES is the same as FOO_SITES you can use somthing like
> BAR_SITES=${FOO_SITES} of course.
Yes, it should work, especially if we don't stick to the binary solution:
there are only ~1000 distinfo with more than one distfile and ~350 with more
than two. So, we can embed distfile information for majority of ports
and keep 350 distinfo files. This will allow us to remove 24k+ files. 

> pkg-descr and pkg-plist will be around for a while I think but maybe pkg
> could support an alternative syntax that is similar to the manifest file
> included in the package.  This file (named Manifest maybe?) would include
> a comment, description, install/deinstall scripts and a list of files and
> directories like this (copied from https://github.com/freebsd/pkg):
I don't like embedded files, they are usually not user-serviceable.
We can add WWW variable to the Makefile though, it will greatly speed up
INDEX building.


-- 
Alex