[FreeBSD-Ports-Announce] [HEADSUP] Staging, packaging and more

[Baptiste Daroussin]

Hi all,

I want to give to all porters infomations on the rationnale behind all the changes,
to explain why some things has happened, the rationale behind what we still need
to do to get the ports tree into a modern binary world.

1/ Staging

You may has notice that staging has hit the ports tree, staging is something
really important, all packages system are using that feature for eons, sometime
called DESTDIR sometime called FAKEDIR.

Staging is consistent in adding a new step while building packages: install
everything into ${STAGEDIR}. Then we can directly create packages out of that
directory without having to install into /. What the implementation does is:
With pkg_install (legacy package tools):
- Create a package from the stage directory
- Install that package.

With pkgng:
- Create a package from that stage directory
_OR_
- Install to / (pkgng can consider the stage directory as if it is a package)

What it means is, that it is the end of the bad plist, only things in plist are
really installed, this is the end of the broken packages that break the system
because they fail in the middle of make install as a package will only be
installed once we are sure the build process properly pass.

It allows right now to build and create a package without the need to be root.
It gives us new targets:
- make makeplist (to create the pkg-plist)
- make check-orphans (to print what is in the stage directory but not in
  pkg-plist)

It reduces code duplication: in the end the installation is being done via a
package installation meaning that PKG-MESSAGE is automatically printed, all
pre/post installation scripts are executed.

It reduces patches: no need anymore to patch upstream Makefiles to avoid
installing the DOCS just do not list them in pkg-plist or conditionnaly list
them in the pkg-plist.

This also allows lots of new features to come:
- Allow to create sub-packages
- Allow to create debuginfo packages.
- Allow to do a lot of sanity check in the staging area to improve our QA

To convert your ports refer to: https://wiki.freebsd.org/ports/StageDir

Along with stage you have noticed that MAN* and MLINKS has gone, and now all the
manpages should be added to the plist. BTW MANCOMPRESSED is also not necessary
now.

There is 3 reason behind making it go and not being replaced:
- The initial goal of those macros was to be able to compress the right manpages
  and to handle the different hardlinks/symlinks between those pages. Because it
  was directly installed in / rather than using a stage directory, it was
  necessary to at a point list them. and to properly track the different links
  in MLINKS (which wasn't done properly in most ports btw :))
  With stage, we have a new compress-man which does it properly on its own
  without the need to get the list of the manpages, without the need of an
  explicit declaration of the links.
  This syntax to handle localized man pages was also terrific :)
  and if you have a port mixing manpages in "non regular" and "regular"
  localtion, it was totally messing
- Consistency, a port is basically: some metadata, a plist and some operations
  to perform. for metadata all metadata are in the Makefile, all operations are
  in the Makefile but plist can be a mix of pkg-plist and Makefile, this is
  inconsistent, it makes sometime hard to figure out why a file has been added
  to the plist etc.
  Also this makes us having tons of targets define to handle those extras
  entries and results in highly inefficient make(1) usage.
- stage is also a first step to go to sub-packages, sub-packages will basically
  be: 1 port able to produce N packages, to be able to do this we will use
  multiple plist, having the files properly defined already in plist will help
  that. Having files defined in macros on Makefile will make it hard to
  determine which one should go in which plist.

Last thing I would like to add about it: I don't see the difference personnaly
between listing N lines of manpages into Makefile MAN* macros where btw you have
to manually define the categories where to put them and adding those N lines
directly into the plist where make makeplist and/or make check-orphans can help
you getting the exact lines automatically?

2/ Binary packages

For the sake of being able to have clean working binary packages, pkgng has had
to use a dirty hack: origin is used has an identifier instead of the package
name.

Why: How to determine than ImageMagick-nox11 and ImageMagick are the same
package? and if I don't know they are the same package how to upgrade them
properly? same goes how the package manager can know that mysql-client-5.2.1 is
not an upgrade of mysql-client-5.0.1?

How can pkgng determine which subversion is the right version to use:
'pkg install subversion' will propose all possible subversion, in that case the
higher version is probably the default version, but in perl case, the higher
version is probably not the one you want.

Some packages are totally different project but have the same package name...
How can pkgng differenciate them.

So please stop using LATEST_LINK to differenciate the different port but
directly make _unique_ package names, pkgng will switch back to package name as
soon as possible (that will also solve the ugly and stupid pkg set -o).

There is multiple possibility to make sure to properly handle multiple versions
of packages:
1/ suffix everything with part of the version (like tcl)
2/ suffix everything but the default with part of the version
3/ have only 2 or 3 different version: project-legacy project project-devel

Please stop renaming the packages base on options! options are now registered
with the package with both pkgng and pkg_install, please stop adding suffix base
on options!

Think about the end user, he will try to install a package, it should be as
transparent for him as possible.

Note that the 2 above are mandatory if you want to provide sub packages and
flavours!

regards,
Bapt
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 196 bytes
Desc: not available
URL: <http://lists.freebsd.org/pipermail/freebsd-ports-announce/attachments/20131003/edd04bb5/attachment.sig>