Indexing FreeBSD Ports with Optional Dependencies

Sun Mar 4 19:28:22 UTC 2012

I am responsible for hundreds of FreeBSD systems with hundreds of different combinations of installed packages.  Back in 2003, I wrote a script that helps me manage this, and over the past few months, I've been doing bug fixes.

My script generates an index of all the ports that I use.  It uses this index to build, install, and update the packages that are used by the hundreds of different systems.  To get the list of each port's dependencies, it runs "make all-depends-list".

I'm having a problem with a few ports that use an automatic detection method to declare their optional dependencies.  With these ports, dependencies and/or other things important to the build process are changed depending on whether or not files from another port exist on the system.  A quick search of the entire ports tree shows that there are less than three hundred ports using this method.  Every other port uses the OPTIONS framework to make these changes, which is easy to use and to write a script around, whereas this automatic method complicates indexing.

Consider this example from the current print/libgnomeprint Makefile:

-----BEGIN QUOTE-----
.include <bsd.port.pre.mk>

.if (defined(WITH_CUPS) || exists(${LOCALBASE}/lib/libcups.so)) && !defined(WITHOUT_CUPS)
CONFIGURE_ARGS+=	--with-cups
LIB_DEPENDS+=		gnomecups-1.0:${PORTSDIR}/print/libgnomecups
PLIST_SUB+=		CUPS:=""
.else
CONFIGURE_ARGS+=	--without-cups
PLIST_SUB+=		CUPS:="@comment "
.endif

pre-everything::
.if !defined(WITH_CUPS)
	@${ECHO_MSG} " Enable cups support by defining"
	@${ECHO_MSG} " WITH_CUPS"
	@${ECHO_MSG} ""
.endif
-----END QUOTE-----

print/libgnomeprint does not use OPTIONS to define WITH_CUPS or WITHOUT_CUPS.  If you haven't defined these as environment variables, running "make all-depends-list" will yield two different results, depending on whether or not you have print/cups-base installed.  print/libgnomeprint will echo "Enable cups support by defining WITH_CUPS" in the "pre-everything" stage, but unless you read through every Makefile before installing each port, it's easy to miss.

I use over a thousand different ports for the hundreds of systems that I maintain.  I'd strongly prefer not to be expected to read through each port's Makefile and/or to stay abreast of every Makefile's changes.  It would be very difficult to write a script to do this for me because of how obscure these Makefiles can be, and I don't have the time for it.

As far as I can tell, the only benefit to using this automatic method is that it slightly reduces the potential number of OPTIONS "dialog" screens that appear.  The reduction across the entire ports tree is only about 1%, so it's barely noticeable.  The cost of this minor benefit is the breaking of indexing, which causes problems for ports management scripts.

What we need is a way to identify which dependencies a port requires, regardless of what other ports have been installed, and without the need to manually read through every Makefile before every build.  I have three solutions to propose:

A. Require ports to use the OPTIONS framework instead of the automatic method that I described above.  This would require an update to the Porter's Handbook and patches to the approximately three hundred ports that currently use this method.  I am willing to write all these changes by myself.

B. Require ports to use the OPTIONS framework in addition to the automatic method that I described above.  This would require an update to the Porter's Handbook and patches to the approximately three hundred ports that currently use this method.  I am willing to write all these changes by myself.

C. Create a new dependency category to be defined by ports that use optional dependencies.  This would require a new type of dependency checking added to bsd.*.mk (perhaps called "OPT_DEPENDS" with "make opt-depends-list"), an update to the Porter's Handbook, and patches to the many thousands of ports that use optional dependencies by any method.  I am not willing to do all of this by myself, and I think it's excessive.

Personally, I like solution A.  Like B, A is significantly simpler than C, but unlike B and C, A eliminates this automatic detection method entirely.

With any of these solutions, the Porter's Handbook would need to be updated.  For solution A, it would forbid port maintainers from using the automatic method for the reasons that I described above.  For solution B, it would explain the need to use the OPTIONS framework whenever the automatic method is used.  For solution C, it would explain the requirement of "OPT_DEPENDS" (or whatever) for inclusion into every port that has optional dependencies.

Using the example from print/libgnomeprint, under solution A, I would change that segment of its Makefile to the following:

-----BEGIN REVISION A-----
OPTIONS=	CUPS "Enable CUPS support" on

.include <bsd.port.pre.mk>

.if defined(WITH_CUPS) && !defined(WITHOUT_CUPS)
CONFIGURE_ARGS+=	--with-cups
LIB_DEPENDS+=		gnomecups-1.0:${PORTSDIR}/print/libgnomecups
PLIST_SUB+=		CUPS:=""
.else
CONFIGURE_ARGS+=	--without-cups
PLIST_SUB+=		CUPS:="@comment "
.endif
-----END REVISION A-----

Under solution B, it would look like this:

-----BEGIN REVISION B-----
OPTIONS=	CUPS "Enable CUPS support" on

.include <bsd.port.pre.mk>

.if (defined(WITH_CUPS) || exists(${LOCALBASE}/lib/libcups.so)) && !defined(WITHOUT_CUPS)
CONFIGURE_ARGS+=	--with-cups
LIB_DEPENDS+=		gnomecups-1.0:${PORTSDIR}/print/libgnomecups
PLIST_SUB+=		CUPS:=""
.else
CONFIGURE_ARGS+=	--without-cups
PLIST_SUB+=		CUPS:="@comment "
.endif
-----END REVISION B-----

And under solution C, it would look like this:

-----BEGIN REVISION C-----
OPT_DEPENDS=	gnomecups-1.0:${PORTSDIR}/print/libgnomecups

.include <bsd.port.pre.mk>

.if (defined(WITH_CUPS) || exists(${LOCALBASE}/lib/libcups.so)) && !defined(WITHOUT_CUPS)
CONFIGURE_ARGS+=	--with-cups
LIB_DEPENDS+=		gnomecups-1.0:${PORTSDIR}/print/libgnomecups
PLIST_SUB+=		CUPS:=""
.else
CONFIGURE_ARGS+=	--without-cups
PLIST_SUB+=		CUPS:="@comment "
.endif

pre-everything::
.if !defined(WITH_CUPS)
	@${ECHO_MSG} " Enable cups support by defining"
	@${ECHO_MSG} " WITH_CUPS"
	@${ECHO_MSG} ""
.endif
-----END REVISION C-----

If anyone has any other ideas or comments, I'm very interested in hearing from you.  My thanks to the FreeBSD Ports Management Team for all the suggestions and feedback you've given me so far.  Your time is greatly appreciated.

-- 
A.J. Kehoe IV (Nanoman)     |  /"\  ASCII Ribbon Campaign
Nanoman's Company           |  \ /   - No HTML/RTF in E-mail
E-mail: nanoman at nanoman.ca  |   X    - No proprietary attachments
WWW: http://www.nanoman.ca/ |  / \   - Respect for open standards
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3855 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-ports/attachments/20120304/28e1bfea/smime.bin