Request for Features: Ports Re-engineering

[Alejandro Pulver]

On Mon, 17 Dec 2007 07:48:07 -0600
Stephen Montgomery-Smith <stephen at math.missouri.edu> wrote:

> > In order to finalize the general nature of the ports re-engineering
> > effort this thread is designed to specifically explore the features
> > people want to see.
> > 
> > Please supply one or more of the following:
> > 
> > * Up to 4 user stories of how you would like to use the ports system
> > (may or may not be possible in the current one)
> > * Up to 4 use cases for how you would like to interact with the ports
> > system
> > * Up to 4 features/requirements you think the new system should have
> > * Up to 4 features/requirements that you feel *MUST* remain from the
> > current system
> > * Any non-functional requirements you can think of
> 
> One thing to look at is package building.  I don't know how they build 
> the official packages, but my guess is that they build each one from a 
> clean system.  But, for example, you have tons of little ports each 
> depending on xorg-server, and to rebuild xorg-server for each little 
> port must be a real burden.
> 

The packages are built in a clean system, which is a chroot environment
(i.e. everything in / from a base install) recreated each time a port
is built. Packages are used, they aren't rebuilt each time. See:

/usr/ports/Tools/portbuild: the scripts running in pointyhat
http://tinderbox.marcuscom.com/: based and synchronized with the other,
but preferable for "user" builds (web interface, etc).

> On the other hand some ports really need to be built from a clean 
> system.  Some of them autodetect ports that are already installed, and 
> then change options appropriately.  (Maybe some of the multimedia ports 
> like vlc do this.)  My guess is that this is to some extent unavoidable 
> because the "configure" script in the port build process probably does 
> this as well.  Anyway, perhaps this autodetecting of ports to provide 
> options needs to be built into the system in a systematic manner.  Then 
> robotic package builders could be trained to glean this information from 
> the build tree (what you refer to as the DAG - is that "directed 
> something graph"?).
> 

Auto-detection is certainly avoidable. Some for example only enable
detection of MMX/SSE/etc instructions when not building in
pointyhat/tinderbox. IIRC ports should respect the users' choice, but
it's not easy with the current OPTIONS handling (some have knobs that
can be set to on/off/auto).

I think this could be solved (for both current and possible new system)
like it's done with Python/wxWidgets/Apache/etc where there are port
preference/user preference/auto detection/system default, in a properly
fallback order. The problem is that there is no framework to do that
with OPTIONS for individual ports.

> I also use packages for my private system.  I have one fast computer 
> where I build all my ports, and then make packages from them to 
> distribute to the other computers.  But I use "pkg_create."  The 
> disadvantage of this is that you don't get all the candy (e.g. the 
> messages in pkg-message, etc).
> 

The messages in pkg-message are packaged with the description/etc in
the generated package. However some ports just print text to the
screen, and that isn't recorded. It mostly depends on the port, but a
recording framework may be useful (i.e. echo to screen and pkg-message).

> I used to build my ports, then do a massive "make package" on all the 
> built ports.  But "make package" is designed so that it only works 
> properly if done before any other port is installed.  This is because 
> you might add autodependencies to the package which were not part of the 
> original build.  "make package" could actually get this dependency 
> information from /var/db/pkg, but it doesn't.  (I filed a PR to effect 
> this change, but no-one else felt it was important.  In particular, 
> space is a premium on bsd.ports.mk because the more you add to it, the 
> longer things like "make index" take.)
> 

That is a complicated problem (that some information is get from the
package database, and other from the ports tree). It generally forces
to have some configuration in a file like /etc/make.conf, because in
some situations it must be the same for building dependencies.

For example if you have a port which requires a different version of
python than default, and a few python modules built with that version,
in pointyhat/tinderbox it won't work because will think the original
port needs the default python version as well as its dependencies (this
is taken from INDEX). On the other hand, when building a port locally
it works (however portupgrade tries to update it the wrong way).

Best Regards,
Ale
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-ports/attachments/20071217/5a06b283/signature.pgp