Port system "problems"

[Matthew Seaman]

On 26/06/2012 12:07, Marcus von Appen wrote:
>> Sub-ports should be much more efficient, as there's a lot of the work
>> involved in installing which is a one-time thing when installing port
>> plus some collection of sub-ports.

> How does that look like in detail? Are there any concepts on about
> how it is supposed to work, what the implications are, and so forth?

Please understand that a lot of this stuff exists only in various
people's heads and is subject to arbitrary change as ideas evolve.
Other parts are more-or-less documented; some is already under
development and may or may not require various degrees of revision in
the light of experience.

Anyhow: the various stages of building a port at the moment are well known:

  1 options dialogue
  2 fetch & verify distfiles (*)
  3 extract (*)
  4 patch (*)
  5 configure (*)
  6 compile
  7 install (*)
  8 post-install scripts
  9 register in pkg DB

Not all ports require all those stages.  The stages marked (*) can
involve resolving various dependencies by installing other ports --
although {FETCH,EXTRACT,PATCH}_DEPENDS are relatively rare, and it's
usually {BUILD,LIB}_DEPENDS before configuring and RUN_DEPENDS before
installation that take up most of the time.

You have to do each of these steps (where applicable) for each port you
install.  There will be some time savings for installing several closely
related ports.  For instance: where you have -client and -server slave
ports, you'll generally not have to download any extra distfiles
(although you would still compare all the checksums.) One of those ports
would be usually a BUILD_DEPENDS of the other, so a lot of the
dependency chasing should already be done.

What's different in the new scheme?

  1 options dialogue
  2 fetch & verify distfiles
  3 extract
  4 patch
  5 configure
  6 compile
  7 install to staging directory tree ***
  8 create packages, sub-packages ***
  9 install packages and sub-packages as selected ***

where running post-install scripts is incorporated into stage 9.  (This
also means post-install scripts only have to be implemented once, unlike
currently where you have to implement both a post-install Makefile
target and a post-install script for use with pkgs.)

Stages 1-6 are essentially the same as at the moment: they may differ in
detail due to amalgamating master- and slave-port options etc. but they
only need to happen *once* for the whole collection of port and
sub-ports.  The same sort of dependency chasing will happen as currently.

Installing into a staging directory tree involves installing all the
executables, data, documentation and other files from the port
irrespective of what sub-port they might belong to.   This bit is an
addition compared to the current ports.  Again though, it's a
one-time-for-all step.

Creating packages in the new scheme (ie. assuming pkgng) is a matter of
tarring up selected files from the staging directory along with two
blobs of meta data -- the YAML file containing the runtime dependency
data, package plist, various scripts to be run on
installation/deinstallation and the checksums of all the files, plus
various other information; and an MTREE file for creating the directory
hierarchy to install into.  It's comparable to the current 'fake-pkg'
target in bsd.ports.mk.  This step is repeated for each of the different
sub-packages, packing up different selections of
the staging directory contents each time.  You may well be creating more
{sub-,}packages here than you ultimately intend to install.

Finally, installing the selected package and sub-packages -- this will
be identical to installing pre-compiled packages downloaded from one of
the standard package repositories, and involves unpacking and verifying
the tarball contents, making any necessary directory structure, moving
the files into place, running any scripts as specified in the metadata
and updating the package DB -- now an actual relational database stored
in a single file: ${PKG_DBDIR}/local.sqlite

Whether the extra/different work done in stages 7, 8 and 9 will negate
the savings from only doing stages 1-6 once remains to be seen.  My
prediction is that mostly you'ld come out ahead, but whether you do, and
by how much will vary significantly between individual ports.

	Cheers,

	Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.                   7 Priory Courtyard
                                                  Flat 3
PGP: http://www.infracaninophile.co.uk/pgpkey     Ramsgate
JID: matthew at infracaninophile.co.uk               Kent, CT11 9PW



-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 267 bytes
Desc: OpenPGP digital signature
Url : http://lists.freebsd.org/pipermail/freebsd-ports/attachments/20120626/4c6fdafc/signature.pgp