ports and dependency hell
julian at freebsd.org
Tue Feb 7 18:04:16 UTC 2017
This is a serious post on a serious issue that ports framework people
seem unaware of.
It' getting too easy to get into dependency hell here (I've spent the
last week fighting this)
In a production system we have to choose packages from different eras.
This is because on an average product different teams are responsible
for different parts of the product, and they all have their own
requirements. Not to mention the stuff that comes in from third party
vendors which "must use version X of bar and Version Z of foo". This
is something that is a fact of life in commercial vendors. Ports needs
to support it, and fast because currently ports is a reason to switch
to Linux. (ammunition for the Linux fanboys). We are only staying for
the ZFS support but that reason will evaporate soon.
We may need node.js 6.95 and a particular version of an apache mod for
example, and a specific version off npm, which all only appeared in
the ports tree at different times, so either we completely ditch the
ports tree all together and import buildroot2 , which allows every
package to have its own revision (but is Linux centric), or we need to
generate frankenports trees. My curent iteration has 359 different
packages, so you an imagine the hilarity if I need to slide 20 of
those back or forth, all independently.
There doesn't seem to be any understanding of this fact from the ports
framework, and it means one has to keep one's own ports tree in source
control, as a branch off the FreeBSD one. (maybe I should look at
the problem is that the internal APIs of ports are changing too much.
If you are going to change the API, then you need to be able to
declare the version separately, maybe in a version/distinfo file that
can be pulled in separately at a different rev, rather than having it
built into the main Makefile of each port. Maybe the Makefile
specifies a revision range it can be used with, but that would make a
huge improvement right there.
You can not just slide one port up by 3 months, and another down by 3
months to get the revs one needs because the damned Mk files have
changed. If I coudl leave the bulk of the Makefile alone and just
slide the 'distingo/rev' file, (or be able to select a rev from one
htat gives many alternatives, that woudl be "wonderful".
Please, ports framework people, think about how this can be done, If
I have to edit a file, the game is lost.
Can we please get some understanding from ports people that they are
making things REALLY HARD on vendors and it really strengthens the
hands of the "ditch FreeBSD and go to Linux" crowd when I need to
spend a whole week trying to integrate in a set of 5 new ports into
The call "It just works under linux, select the versions you want of
each package and type make" is often heard around the company. And
management is not totally deaf.
If you want to see how its' done better (still not perfect), go build
a busybox system. you can effectively select any version of any tool
and they all communicate via the pkgconf mechanism and you get the
result you want.(mostly). And the API is stable.
On the pkg side of things we need the ability for pkg to say "yeah I
know I'm looking for foo-1.2.3.txz as a perfect match, but I've been
given some slack on the third digit and I can see a foo-1.2.1.txz, so
I'll install that instead".
Otherwise we just have to spend WAY too much time generating dozens of
"matching sets" of packages , that must be kept around forever if just
one machine shipped with that set, not to mention the fact that making
the matching set is often a real task.
The way to get around the problem above CAN be (not always) to install
foo.1.2.1 first and THEN install the package you actually want, and
pkg will accept it. The problem comes when pkg needs to install a
dependency itself. Then it becomes "super picky", when there is
actually a range of package revisions that would do. Instead of
letting pkg install what it needs, we need to manually set up scripts
to install the dependencies. so that all the work done by pkg is wasted.
Please think about these two issues..
More information about the freebsd-ports