To all port maintainers: libtool

[Tijl Coosemans]

On Fri, 06 Jun 2014 09:54:37 -0500 Bryan Drewery wrote:
> On 6/6/14, 9:27 AM, Tijl Coosemans wrote:
>> On Fri, 06 Jun 2014 15:02:24 +0200 Alexander Leidinger wrote:
>>> Quoting Tijl Coosemans <tijl at freebsd.org> (from Thu, 5 Jun 2014
>>> 18:53:03 +0200):
>>>> On Thu, 05 Jun 2014 09:39:21 -0500 Bryan Drewery wrote:
>>>>> I don't know what .la files are used for and have no time currently to
>>>>> research it.
>>>>>
>>>>> What is the impact to non-ports consumers of removing .la files? Do they
>>>>> also need patches to make them build?
>>>>
>>>> Removing a .la file is somewhat like a library version bump.  Anything
>>>> that depends on it needs to be recompiled.
>>>
>>> I remember from tests waaaaay in the past that not all programs will
>>> be happy when the .la files are not there. I remember that I once
>>> tried to remove the .la files but it didn't work as the program wanted
>>> to open the .la files (after recompile). Maybe libltdl is openening
>>> them? Did you make some checks/tests in this regard?
>>
>> Essentially .la files are small shell scripts that set some variables so
>> in theory they can be used in all kinds of places, but this seems of
>> little practical value.  Libltdl can open and parse .la files (to find
>> the name of the .so file it can dlopen) but it can also work directly
>> with .so files.
>>
>> If a program uses .la files directly then the port can't delete them of
>> course, but so far I haven't encountered such programs.
> 
> My main question was non-ports consumers though. What is the impact on 
> them? We're talking a lot about bumping revisions and forcing rebuilds.
> 
> Will non-ports consumers require rebuild or changes if .la files are
> missing?

If they have a .la file that links to the .la file that went missing
then their .la file needs to be rebuilt.

Like I said before, this is very similar to .so version bumps.  If they
have a program (or library) that links to another library that goes
from .so.X to .so.(X+1) then their program (or library) needs to be
rebuilt too.

This is nothing out of the ordinary.  If you update a dependency that
exists in the ports tree you might have to rebuild stuff that exists
outside the tree.  This is something you always have to keep in mind.

> Would these non-ports consumers be able to properly build and link
> without .la files and without modifications? These .la files seem
> similar to .pc files which are often critical to out-of-tree consumers
> that assume Linux /usr paths instead of /usr/local paths.

You specifically asked about the impact of the removal of .la files so
I restricted my answer to just that, but now I suspect you really want
to know about all impacts of USES=libtool.  So here it goes.

A .la file operates one level below a .pc file so to speak.  A libfoo.pc
file gives you information about a libfoo package (what compiler and
linker flags to use to link with libfoo), while a libfoo.la file will
give you information about the dependencies of libfoo.  Currently all
forms of USES=libtool erase this dependency information or erase the
.la file altogether.

Suppose you have a program (or library) X outside the ports tree which
links to libfoo inside the tree which in turn links to libbar.  The
command to link X looks roughly like this:

libtool --mode=link cc -o X X1.o X2.o -L/usr/local/lib -lfoo

Without USES=libtool in the libfoo port, the libtool script will turn
that cc command into "cc -o X X1.o X2.o -L/usr/local/lib -lfoo -lbar"
and the resulting binary X will link to both libfoo.so and libbar.so.
If X is a library, libtool will also generate X.la which lists libfoo.la
and libbar.la as dependencies.  Note that these dependencies accumulate.
A binary Y that links to library X ends up linking to X.so, libfoo.so
and libbar.so.  That's what we want to get rid of.

With USES=libtool (any form) in the libfoo port, there's no dependency
information in libfoo.la so the above libtool command does not add -lbar
to the cc command line.  X only links with libfoo.so and X.la only lists
libfoo.la as a dependency (if it exists, otherwise just -lfoo).

This may fail if X uses libbar API directly.  The modification you have
to make then is to add libbar as a dependency of X in your makefile.
USES=libtool exposes these kinds of hidden dependencies.  It forces
direct dependencies to be listed explicitly.

So far the case of libfoo being a shared library.  Things may break if
it's a static library.  If a program X uses functions from a file in the
static libfoo.a archive and that file requires functions from libbar then
you must also specify -lbar on the command line so it was very convenient
for libtool to add -lbar automatically.  With USES=libtool (any form) in
the libfoo port you must now add libbar as a dependency of X in your
makefile even if X doesn't use libbar API directly.

We may always revisit this of course but at the time of the creation of
USES=libtool this was considered an acceptable loss for the following
reasons:

1) Static linking with libtool outside the ports tree is assumed to
be used rarely and if it's used it won't be with any complicated
dependency chains.
2) The accumulated overlinking of shared libraries is too bad to ignore.
3) All major Linux distributions do this too.
4) There's an alternative with pkg-config.  It can add -lbar automatically
too.  If the libfoo port installs a .pc file "pkg-config --libs --static
libfoo" will expand to "-L/usr/local/lib -lfoo -lbar".

So to summarise, when a port adds some form of USES=libtool, code outside
the tree may see these two errors:

1) Linking X fails due to a missing .la file.
Solution: find the .la file that still contains a reference to the missing
.la file (simple grep will do) and rebuild the library it corresponds
to.

2) Linking X fails due to unresolved symbols.
Solution: add the libraries that provide those symbols to the X linker
command line in your makefile (or use "pkg-config --libs --static" instead
of "pkg-config --libs").