shared library loader configuration
no at spam at mgedv.net
nospam at mgedv.net
Mon Jul 10 13:07:16 UTC 2006
> -----Original Message-----
> From: owner-freebsd-questions at freebsd.org
> [mailto:owner-freebsd-questions at freebsd.org] On Behalf Of
> Giorgos Keramidas
> Sent: Friday, July 07, 2006 3:46 PM
> To: nospam at mgedv.net
> Cc: freebsd-questions at freebsd.org
> Subject: Re: shared library loader configuration
>
> On 2006-07-07 14:58, "no at spam@mgedv.net" <nospam at mgedv.net> wrote:
> >Giorgos Keramidas wrote:
> >>On 2006-07-07 14:22, "no at spam@mgedv.net" <nospam at mgedv.net> wrote:
> >>> dunno, if it's a misunderstanding, but my only question
> "how to tell
> >>> the system where to load libraries and in which order to prefer
> >>> paths" seems to be still open.
> >>>
> >>> anyway, thx for the reply ;-)
> >>>
> >>> ps: i already RdTFM ;-)
> >>
> >> You don't. Unless you modify the /etc/rc.d/ldconfig
> script manually,
> >> /lib and /usr/lib will always be the first to search.
> >>
> >> I'm still not convinced that "telling the system where to load
> >> libraries from" is the solution to you problem, but I
> don't know what
> >> the problem is. You have to describe first *WHAT* the real problem
> >> is and *WHY* you think modifying the library path is a solution.
> >
> > i found the ldconfig rc-script but i thought there might be a
> > "cleaner" way of telling the system where the shared
> libraries are to
> > be found.
> >
> > any way to tell the system: take /usr/local/lib first w.o. changing
> > the ldconfig rc-scripts or developing own startup scripts
> that achieve
> > that? no way of changing some default configuration file that is
> > avail. for that purpose?
>
> No. For a very good reason too. If you change the default
> loader path
> to use /usr/local/lib first then *ALL* the programs are
> affected. Even
> those that are part of the base system. This is, in general,
> a very bad
> idea as the base-system programs may depend on particular versions of
> the libraries to work correctly and those libraries are in
> /lib:/usr/lib.
>
> > some additional thoughts (a little bit of phil.):
> >
> > i wonder, that anybody scripts such hardcoded stuff into a script
> > because the environment /etc/ld*conf* exists, and at least
> for a clear
> > and proper way for the admin to define what to load from where it
> > should be possible, to override a default configuration via the
> > config-files, and not with modifications to rc-scripts
> which are gone
> > by default after each upgrade.
>
> There's a reason why /lib:/usr/lib take precedence over /usr/local.
>
> Don't change that, or be prepared to debug all the potential bugs that
> may start appearing.
>
> > to satisfy your couriosity :-)
>
> This description is much better.
>
> > i'd like to compile openssl 0.9.8 and a newer zlib for testing some
> > software that does crypto & compression using these libs. and i
> > wanted to keep the servers as clean as possible from changing
> > rc-scripts, etc... to ensure we're able to transfer the outcoming
> > piece of program to other boxes w'out much effort. i know
> it's inside
> > the ports but the problem is, we'd like to tes some sort of code
> > that's not enabled by default in the ports.
>
> You can always install your own version of openssl and zlib and use an
> explicit -R option to tell the linker where to find libraries. There
> are two steps involved in linking and distributing an executable:
>
> * Build-time linking, where the compiler/linker has to be
> explicitly
> pointed at teh right place with -L/foo/lib paths.
>
> * Runtime linking, where the runtime linker-loader has to
> lookup and
> locate the library.
>
> You can affect the first one with explicit -L/foo/lib
> options. You can
> also hardcode one or more paths to the binary -- solving the second
> problem too -- if you use -R/distrib/lib options while building.
>
> Your problems are only a matter of using the proper build options.
>
> See for example the ldd output near the end of the following log:
>
> # root at gothmog:/home/giorgos/tmp/foo# make cleandir
> # ===> lib (cleandir)
> # ===> lib/libz (cleandir)
> # rm -f a.out foo.o foo.o.tmp
> # rm -f foo.po foo.po.tmp
> # rm -f foo.So foo.so foo.So.tmp
> # rm -f libz.so
> # rm -f libz.so.* libz.so
> # rm -f libz.a libz_p.a libz.so.1
> # rm -f .depend GPATH GRTAGS GSYMS GTAGS
> # ===> bin (cleandir)
> # ===> bin/foo (cleandir)
> # rm -f foo foo.o
> # rm -f .depend GPATH GRTAGS GSYMS GTAGS
> # root at gothmog:/home/giorgos/tmp/foo# make cleandir
> # ===> lib (cleandir)
> # ===> lib/libz (cleandir)
> # rm -f a.out foo.o foo.o.tmp
> # rm -f foo.po foo.po.tmp
> # rm -f foo.So foo.so foo.So.tmp
> # rm -f libz.so
> # rm -f libz.so.* libz.so
> # rm -f libz.a libz_p.a libz.so.1
> # rm -f .depend GPATH GRTAGS GSYMS GTAGS
> # ===> bin (cleandir)
> # ===> bin/foo (cleandir)
> # rm -f foo foo.o
> # rm -f .depend GPATH GRTAGS GSYMS GTAGS
> # root at gothmog:/home/giorgos/tmp/foo# make obj
> # ===> lib (obj)
> # ===> lib/libz (obj)
> # /home/giorgos/tmp/foo/obj/home/giorgos/tmp/foo/lib/libz
> created for /home/giorgos/tmp/foo/lib/libz
> # ===> bin (obj)
> # ===> bin/foo (obj)
> # /home/giorgos/tmp/foo/obj/home/giorgos/tmp/foo/bin/foo
> created for /home/giorgos/tmp/foo/bin/foo
> # root at gothmog:/home/giorgos/tmp/foo# make
> # ===> lib (all)
> # ===> lib/libz (all)
> # cc -O2 -fno-strict-aliasing -pipe
> -I/home/giorgos/tmp/foo/lib/libz -g -Wsystem-headers -Werror
> -Wall -Wno-format-y2k -Wno-uninitialized -c
> /home/giorgos/tmp/foo/lib/libz/foo.c
> # building static z library
> # ranlib libz.a
> # cc -pg -O2 -fno-strict-aliasing -pipe
> -I/home/giorgos/tmp/foo/lib/libz -g -Wsystem-headers -Werror
> -Wall -Wno-format-y2k -Wno-uninitialized -c
> /home/giorgos/tmp/foo/lib/libz/foo.c -o foo.po
> # building profiled z library
> # ranlib libz_p.a
> # cc -fpic -DPIC -O2 -fno-strict-aliasing -pipe
> -I/home/giorgos/tmp/foo/lib/libz -g -Wsystem-headers -Werror
> -Wall -Wno-format-y2k -Wno-uninitialized -c
> /home/giorgos/tmp/foo/lib/libz/foo.c -o foo.So
> # building shared library libz.so.1
> # ===> bin (all)
> # ===> bin/foo (all)
> # cc -O2 -fno-strict-aliasing -pipe
> -I/home/giorgos/tmp/foo/bin/foo/../../lib/libz -g -c
> /home/giorgos/tmp/foo/bin/foo/foo.c
> # cc -O2 -fno-strict-aliasing -pipe
> -I/home/giorgos/tmp/foo/bin/foo/../../lib/libz -g -L-
> -L/home/giorgos/tmp/foo/bin/foo/../../lib/libz
> -L/home/giorgos/tmp/foo/obj/home/giorgos/tmp/foo/bin/foo/../..
> /lib/libz -L/usr/lib -R/tmp/lib -o foo foo.o -lz
> # root at gothmog:/home/giorgos/tmp/foo# make install
> # ===> lib (install)
> # ===> lib/libz (install)
> # install -C -o root -g wheel -m 444 libz.a /tmp/lib
> # install -C -o root -g wheel -m 444 libz_p.a /tmp/lib
> # install -o root -g wheel -m 444 libz.so.1 /tmp/lib
> # ln -fs libz.so.1 /tmp/lib/libz.so
> # ===> bin (install)
> # ===> bin/foo (install)
> # install -o root -g wheel -m 555 foo /tmp/bin
> # root at gothmog:/home/giorgos/tmp/foo# ldd /tmp/bin/foo
> # /tmp/bin/foo:
> # libz.so.1 => /tmp/lib/libz.so.1 (0x2807c000)
> # libc.so.7 => /lib/libc.so.7 (0x2807e000)
> # root at gothmog:/home/giorgos/tmp/foo#
>
> Note how the /tmp/bin/foo executable depends on `libz' but it knows
> where to pick it up from. No conflict with the system version of
> libz.so and certainly no problems when the utility runs:
>
> # root at gothmog:/home/giorgos/tmp/foo# /tmp/bin/foo
> # 1
> # 0
> # 1
> # 0
> # 1
> # 0
> # 1
> # 0
> # 1
> # 0
> # root at gothmog:/home/giorgos/tmp/foo#
>
> The Makefile of this `foo' program and the library it uses
> can be found
> online at:
>
> http://people.freebsd.org/~keramida/files/foo-rtld.tgz
>
> What you can do to override on a _per_program_ basis the location of
> libraries is to use:
>
> * Proper -L/build/path/lib options when compiling to let the
> compiler file the build-time version of the library.
>
> * Proper -R/runtime/path/lib options when compiling to add hints
> about the location of the libraries in the binary.
> This way, the
> runtime linker will know where to look for shared libs.
>
> > my (very subjective) point of view currently is, that the dynamic
> > loading environment could rely on just one (or 2 if you care for
> > elf/aout anymore) configuration file, which is being taken
> care of the
> > libexec-stuff.
>
> It does. These paths are /lib and /usr/lib, and there is no
> good reason
> to change them to satisfy just one program.
>
> > and assume the following:
> > you're compiling ~10 libraries in series (all go to /usr/local) and
> > they're some sort of connected to each other, you'll always have to
> > exec ldconfig -blabla... to ensure, the loader knows about the new
> > lib?
>
> Nope. The libraries should include -R options that point to the right
> place for their dependencies.
>
> It's the responsibility of the builder & packager to make sure these
> thigns are done Right(TM), so you won't have to fiddle with tons of
> conflicting options and LD_LIBRARY_PATH trickery to have
> something that
> sort of works.
>
> Just think about it...
>
> [case 1]
>
> Let's say that today you install something that requires
> /usr/local/lib to be the first, just because the guy who
> compiled libfoo.so thought this was a good idea.
>
> Then you have to install something else, which depends on
> /usr/lib being first because another developer thought this
> was a good idea too (or simpe because his program has been
> built with the system version of the same library).
>
> Suddenly, you can only have *one* of these working.
>
> [case 2]
>
> You install FreeBSD, which comes with a few dozen tools that
> use Gzip compression. FreeBSD comes bundled with a carefully
> tuned, debugged, tested and QA'd version of libz.so -- a
> library which works 100% as expected with the system
> binaries.
>
> Then one day, you install software package Frobnic Express,
> from a company who thought it is a marvelous idea to depend
> on /usr/local being the first library lookup path. You
> switch, and suddenly nothing from the base system works
> correctly. Programs crash, create bogus output files, or
> randomly misbehave. After a week of effort, you find that
> this is because Frobnic Express installed an incompatible
> libz.so in /usr/local/lib which the system binaries can never
> work well with.
>
> It's just not a good idea. Avoid it. At all costs. The hell of
> maintenance effort you will have to go through when things go wrong is
> not nice. Trust me when I say you won't like it at all :-)
>
> > hey, i'm not a real developer (as you can see) and i really
> > am sure that there was a reason for developing that way.
> > i'm also sure, that you can explain good reasons for doing
> > it that way (and i wouldn't ever say "i don't care 'bout that!")
> >
> > but from a more or less "user's" point of view, it's admin-
> > overhead which could (theoretically) be avoided.
> >
> > i'll STFU know - things are going too theoretically ;-)
>
> This may sound a bit harsh, but if you are not a developer,
> why are you
> compiling things?
>
> On the other hand, look at the sample `foo-rtld.tgz' tarball
> I created.
> See how the fake `libz' and the `foo' utility that depend on it work.
> See how they are compiled, and try to use a similar scheme :)
>
> I hope this clarifies a bit why I think you are headed down a
> very wrong
> path by trying to override /lib and /usr/lib's precedence...
>
> Regards,
> Giorgos
>
1st of all, thx for the efforts to clarify things. not
everybody would take this amount of time for a lazy bugger
like me who doesn't like to change env-vars and compiler-
args.
as mentioned, i preferred a simple "switch" that renders the
whole compile thing as a "change it, forget about it" thing.
that this will break up thinks, is obvious. and i already
thought about it (additionally, i thought it might be good
for the os to e.g. use openssl-0.9.8b instead of 0.9.7e,
where many improvements (security and speed) have been made.
but as far as i could figure out, most things are linked
against the specific version and changing ld-paths doesn't
affect anything.
ok dude, you convinced me a little bit and if it were more
effort than changing centralized scripts written by us, i'd
still wanted to change ldconfig ;-)
but now, i use -L and -R, this works fine (until now) and
maybe it's not necessary anymore, to workaround ldconfig.
regards ;-)
More information about the freebsd-questions
mailing list