Multiple logical packages from a single source port

[Marcus von Appen]

Dear all,

now that we removed lang/python from bsd.python.mk, we need to enable
python ports to be built and installed for different python versions
without creating conflicts.
Looking at the TODO list, which I posted ages ago, this should be the
very last thing before the ports tree is able to support different
Python versions at the same time (the pkg tools are not, but that
another issue to deal with).

Many python ports install binaries (or scripts), docs, additional data
and other things besides their modules. While the latter, the modules,
already support different python versions out of the box (since they are
usually installed locally into a version-specific site-packages folder),
the first do not.

DATADIR, DOCDIR, WWWDIR and ETCDIR follow a <PREFIX>...<PORTNAME>
layout, and as such use the same directory for different python
versions. Binaries are often installed as defined by the upstream port
and as thus are not distinct when it comes to different python versions.

To work around this limitation and to enable ports as well as packages
to be installed for different python versions at the same time, the
current standard behaviour has to be tweaked.

I'd propose that standard directories, which currently use the PORTNAME
as identifier, shall carry a python version prefix. Binaries shall carry
a python version suffix, but shall be symlinked to their original name,
if the python version of the port matches the default python version
choice of the user.

What does that mean in practice? Assume, we have port devel/py-foo,
which, at this very moment, uses the following installation layout for
Python 2.7 (being the chosen default of the user):

  bin/foo
  lib/python2.7/site-packages/foo/__init__.py
  lib/python2.7/site-packages/foo/bar.py
  lib/python2.7/site-packages/foo/foofoo.py
  share/foo/bar/some.dat
  share/foo/arbitrary.dat
  share/doc/foo/README

In a prefixed ports world, the installation layout would be:

  bin/foo-py27
  bin/foo [links to]-> bin/foo-py27
  lib/python2.7/site-packages/foo/__init__.py
  lib/python2.7/site-packages/foo/bar.py
  lib/python2.7/site-packages/foo/foofoo.py
  share/py27-foo/bar/some.dat
  share/py27-foo/arbitrary.dat
  share/doc/py27-foo/README

If the user chooses to install devel/py-foo for a non-default python version
3.3, the installation layout would be:

  bin/foo-py33
  lib/python3.3/site-packages/foo/__init__.py
  lib/python3.3/site-packages/foo/bar.py
  lib/python3.3/site-packages/foo/foofoo.py
  share/py33-foo/bar/some.dat
  share/py33-foo/arbitrary.dat
  share/doc/py33-foo/README

If the user would make Python 3.3 the default version and rebuild and
reinstall the port for 2.7 and 3.3, a symlink bin/foo --> bin/foo-py33
would appear instead of bin/foo --> foo-py27.

Certainly docs, data and other shared content would duplicate, but this
is safer, easier to maintain and less error-prone than to use shared
directories and lots of conditional magic in the installation and
deinstallation logic.

I created a POC as USES, named uniquefiles.mk, which can be used by
nearly every port (even non-python ones). For USE_PYDISTUTILS,
it would be implicitly used, other ports, e.g. autotools-based ones
installing python modules, would need to pull in the python specific
paramters via UNIQUE_PYTHON_FILES=yes.

Q: Do I have to touch the plists to enable support for it?

   No. The port logic will do that for you. You must not add a
   prefixed or suffixed binary name, though, but the original name
   only. In short, you MUST NOT add things like

     bin/foo-py27 or
     bin/foo-py32 or
     sbin/foo-py33
     ...

   if the upstream package does not create those files (which is
   unlikely).

Q: So nothing to be done for me?

   Right. Just watch your plists on updating.

Q: UNIQUE_PYTHON_FILES?

   Yes. If you are maintaining a python port, which does not use distutils (no
   USE_PYDISTUTILS=... in the Makefile), UNIQUE_PYTHON_FILES is what you want.
   Ports using USE_PYDISTUTILS will implicitly configure everything, so that
   DOCDIR, DATADIR, ... and binaries will be properly prefixed and suffixed.
   Your python port however does not use USE_PYDISTUTILS, so you can help
   yourself using UNIQUE_PYTHON_FILES=yes.

Q: Can I use it for my own port? It does not use python, though...

   Of course. uniquefiles.mk is a USES, thus each and every port can utilise
   it. Please refer to its inline comments for further details about how to
   configure and use it.

A patch for the current ports tree can be found at
http://people.freebsd.org/~mva/python_unique_ports.diff

If you just want to try out the uniquefiles feature, you can find a
separate diff at
http://people.freebsd.org/~mva/uniquefiles.diff

Readers are encouraged to give those patches a try in an own jail
:-). So far, I only tested them briefly, but will do more tests the next
days.

Cheers
Marcus
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 196 bytes
Desc: not available
URL: <http://lists.freebsd.org/pipermail/freebsd-python/attachments/20131219/021d51cd/attachment.sig>