svn commit: r270803 - head/libexec/rtld-elf
Bryan Drewery
bdrewery at FreeBSD.org
Fri Aug 29 19:46:08 UTC 2014
On 8/29/2014 12:23 PM, Konstantin Belousov wrote:
> On Fri, Aug 29, 2014 at 09:57:31AM -0500, Bryan Drewery wrote:
>> On 8/29/2014 5:44 AM, Konstantin Belousov wrote:
>>> Author: kib
>>> Date: Fri Aug 29 10:44:58 2014
>>> New Revision: 270803
>>> URL: http://svnweb.freebsd.org/changeset/base/270803
>>>
>>> Log:
>>> Document the whole settings needed to build a debug version of rtld.
>>>
>>> Sponsored by: The FreeBSD Foundation
>>> MFC after: 3 days
>>>
>>> Modified:
>>> head/libexec/rtld-elf/Makefile
>>>
>>> Modified: head/libexec/rtld-elf/Makefile
>>> ==============================================================================
>>> --- head/libexec/rtld-elf/Makefile Fri Aug 29 10:43:56 2014 (r270802)
>>> +++ head/libexec/rtld-elf/Makefile Fri Aug 29 10:44:58 2014 (r270803)
>>> @@ -1,5 +1,9 @@
>>> # $FreeBSD$
>>>
>>> +# Use the following command to build local debug version of dynamic
>>> +# linker:
>>> +# make DEBUG_FLAGS=-g DEBUG=-DDEBUG MK_TESTS=no all
>>> +
>>> .include <src.opts.mk>
>>> MK_SSP= no
>>>
>>>
>>
>> How difficult would it be to allow DEBUG to be set during runtime like
>> GNU's can with LD_DEBUG? I have found GNU's LD_DEBUG to be very useful
>> for userland debugging, especially when using dlopen(3).
>>
>> We have LD_DEBUG environment variable but it only prints if built with
>> -DDEBUG.
>>
>> Is there a concern about performance by enabling this based only on
>> environment?
>
> I am sure that nobody evaluated the performance consequences of
> unconditionally compiling the debugging stuff in. I am sure that the
> ld-elf.so.1 size will increase.
>
> The reason why nobody cares to evaluate and enable this stuff by default
> is that the debugging output is very ad-hoc. I found it almost useless
> in both the content and amount of data it outputs. The reason why I
> enable it for my work on ld-elf is that I insert my own dbg() calls for
> debugging (and usually remove them before the commit since they add even
> more verbosity useless for general public consumption).
Ah.
>
> I find the combination of the ELF dumping tools like readelf and
> objdump, together with gdb (-g) and custom dbg() statement (slighly
> glorified printf debugging) most adequate combination to debug rtld.
>
> After the long preamble. What use do you have for LD_DEBUG ? If it is
> possible to formalize and tailor the debugging output for real users
> needs, I am all for making it available unconditionally from LD_DEBUG
> knob. The significants part of the current dbg() statements would be
> removed or require more agressive settings to become active.
>
My only uses so far have been observing which file is loaded for a
library with dlopen(3) and where symbols are resolved from. Seeing where
symbols are resolved from is useful for both dlopen(3)/dlsym(3) usage
and LD_PRELOAD.
I've written an application that "optionally" allows using some
libraries such as libtcl to be used without requiring the library at
build time or startup. At build time the library's headers are used to
generate wrapper functions for every symbol needed. The library is not
linked in. When the binary is ran it starts up fine if libtcl is
missing. At runtime if TCL is enabled then it will attempt to
dlopen(libtcl.so.VER) using the VER its symbols was compiled against. If
it cannot find it then the feature is disabled. It uses dlsym(3) to load
all symbols into a table and the wrapper functions use that table via
hash table lookups.
This all pretty convoluted and only done to avoid requiring a library at
startup. The application is distributed pre-compiled for specific OS
releases and not intended to be compiled manually.
Anyway some examples from GNU's rtld:
> # env LD_DEBUG=help ./app
> Valid options for the LD_DEBUG environment variable are:
>
> libs display library search paths
> reloc display relocation processing
> files display progress for input file
> symbols display symbol table processing
> bindings display information about symbol binding
> versions display version dependencies
> all all previous options combined
> statistics display relocation statistics
> unused determined unused DSOs
> help display this help message and exit
>
> To direct the debugging output into a file instead of standard output
> a filename can be specified using the LD_DEBUG_OUTPUT environment variable.
With libs:
> # env LD_DEBUG=libs ./app
> ...
> 9328: find library=libtcl.so [0]; searching
> 9328: search cache=/etc/ld.so.cache
> 9328: search path=/lib64/tls/x86_64:/lib64/tls:/lib64/x86_64:/lib64:/usr/lib64/tls/x86_64:/usr/lib64/tls:/usr/lib64/x86_64:/usr/lib64 (system search path)
> 9328: trying file=/lib64/tls/x86_64/libtcl.so
> 9328: trying file=/lib64/tls/libtcl.so
> 9328: trying file=/lib64/x86_64/libtcl.so
> 9328: trying file=/lib64/libtcl.so
> 9328: trying file=/usr/lib64/tls/x86_64/libtcl.so
> 9328: trying file=/usr/lib64/tls/libtcl.so
> 9328: trying file=/usr/lib64/x86_64/libtcl.so
> 9328: trying file=/usr/lib64/libtcl.so
> 9328:
> 9328: find library=libpthread.so.0 [0]; searching
> 9328: search path=/usr/lib64 (system search path)
> 9328: trying file=/usr/lib64/libpthread.so.0
> 9328: search cache=/etc/ld.so.cache
> 9328: trying file=/lib64/libpthread.so.0
> 9328:
> 9328:
> 9328: calling init: /lib64/libpthread.so.0
> 9328:
> 9328:
> 9328: calling init: /usr/lib64/libtcl.so
> 9328:
> 9328: find library=libnss_compat.so.2 [0]; searching
> 9328: search cache=/etc/ld.so.cache
> 9328: trying file=/lib64/libnss_compat.so.2
> 9328:
> 9328: find library=libnsl.so.1 [0]; searching
> 9328: search cache=/etc/ld.so.cache
> 9328: trying file=/lib64/libnsl.so.1
> 9328:
> 9328:
> 9328: calling init: /lib64/libnsl.so.1
> 9328:
> 9328:
> 9328: calling init: /lib64/libnss_compat.so.2
> 9328:
> 9328: find library=libnss_nis.so.2 [0]; searching
> 9328: search cache=/etc/ld.so.cache
> 9328: trying file=/lib64/libnss_nis.so.2
> 9328:
> 9328: find library=libnss_files.so.2 [0]; searching
> 9328: search cache=/etc/ld.so.cache
> 9328: trying file=/lib64/libnss_files.so.2
> 9328:
> 9328:
> 9328: calling init: /lib64/libnss_files.so.2
> 9328:
> 9328:
> 9328: calling init: /lib64/libnss_nis.so.2
> ^C
> 9328: calling fini: /usr/lib64/libtcl.so [0]
> 9328:
> 9328:
> 9328: calling fini: /lib64/libm.so.6 [0]
> 9328:
> 9328:
> 9328: calling fini: /lib64/libdl.so.2 [0]
> 9328:
> 9328:
> 9328: calling fini: /lib64/libpthread.so.0 [0]
> 9328:
> 9328:
> 9328: calling fini: /lib64/libnss_compat.so.2 [0]
> 9328:
> 9328:
> 9328: calling fini: /lib64/libnss_nis.so.2 [0]
> 9328:
> 9328:
> 9328: calling fini: /lib64/libnsl.so.1 [0]
> 9328:
> 9328:
> 9328: calling fini: /lib64/libnss_files.so.2 [0]
> 9328:
> 9328:
> 9328: calling fini: /lib64/libc.so.6 [0]
With symbols:
> # LD_DEBUG=symbols ./app
> ...
> 9355: symbol=Tcl_SetObjResult; lookup in file=/lib64/libdl.so.2 [0]
> 9355: symbol=Tcl_SetObjResult; lookup in file=/usr/lib/gcc/x86_64-pc-linux-gnu/4.8.2/libstdc++.so.6 [0]
> 9355: symbol=Tcl_SetObjResult; lookup in file=/lib64/libm.so.6 [0]
> 9355: symbol=Tcl_SetObjResult; lookup in file=/usr/lib/gcc/x86_64-pc-linux-gnu/4.8.2/libgcc_s.so.1 [0]
> 9355: symbol=Tcl_SetObjResult; lookup in file=/lib64/libc.so.6 [0]
> 9355: symbol=Tcl_SetObjResult; lookup in file=/lib64/ld-linux-x86-64.so.2 [0]
> 9355: symbol=Tcl_SetObjResult; lookup in file=/usr/lib64/libcrypto.so.1.0.0 [0]
> 9355: symbol=Tcl_SetObjResult; lookup in file=/lib64/libz.so.1 [0]
> 9355: symbol=Tcl_SetObjResult; lookup in file=/usr/lib64/libtcl.so [0]
> 9355: symbol=Tcl_GetObjResult; lookup in file=./netplay [0]
> 9355: symbol=Tcl_GetObjResult; lookup in file=/lib64/libdl.so.2 [0]
> 9355: symbol=Tcl_GetObjResult; lookup in file=/usr/lib/gcc/x86_64-pc-linux-gnu/4.8.2/libstdc++.so.6 [0]
> 9355: symbol=Tcl_GetObjResult; lookup in file=/lib64/libm.so.6 [0]
> 9355: symbol=Tcl_GetObjResult; lookup in file=/usr/lib/gcc/x86_64-pc-linux-gnu/4.8.2/libgcc_s.so.1 [0]
> 9355: symbol=Tcl_GetObjResult; lookup in file=/lib64/libc.so.6 [0]
> 9355: symbol=Tcl_GetObjResult; lookup in file=/lib64/ld-linux-x86-64.so.2 [0]
> 9355: symbol=Tcl_GetObjResult; lookup in file=/usr/lib64/libcrypto.so.1.0.0 [0]
> 9355: symbol=Tcl_GetObjResult; lookup in file=/lib64/libz.so.1 [0]
> 9355: symbol=Tcl_GetObjResult; lookup in file=/usr/lib64/libtcl.so [0]
I would love to work on adding such a feature to ours, especially if it
can assist with debugging rtld itself. I would think with some
predict_false() checks we could avoid much of the performance penalties.
As a side note I think I may have found an rtld issue (on 8.4) with -32
handling. Being a production machine I don't want to attempt installing
a DEBUG rtld to figure out the problem. Having LD_DEBUG would have been
useful on this as well.
The issue I ran into with -32 was from converting a i386 system to
amd64. I have hundreds of out-of-ports binaries that need to be manually
recompiled. To allow delaying this task for a bit I copied all of the
libraries from /usr/local/lib from the old i386 system to
/usr/local/lib32/compat/system and added all of the paths in there to
ldconfig:
> [~] # ldconfig -32 -r|head -n3
> /var/run/ld-elf32.so.hints:
> search directories: /usr/lib32:/usr/local/lib32/compat/system:/usr/local/lib32/compat/system/mysql:/usr/local/lib32/compat/system/gnutls3:/usr/local/lib32/compat/system/pth:/usr/local/lib32/compat/system/qt4:/usr/local/lib32/compat/system/tdom0.8.3:/usr/local/lib32/compat/system/gcc46:/usr/local/lib32/compat/system/gcc47:/usr/local/lib32/compat/system/gcc48:/usr/local/lib32/compat/system/gcc49:/usr/local/lib32/compat/system/event2:/usr/local/lib32/compat/pkg:/usr/local/lib32/compat
It worked for 90% of the system but then I ran into some libraries that
refused to find their own dependencies. The issue seemingly is that
dependencies for libraries that are also in the -32 path are not looked up.
For example:
> [~] # ldd -a /usr/local/lib32/compat/system/libssl.so
> /usr/local/lib32/compat/system/libssl.so:
> libcrypto.so.8 => not found (0x0)
> libthr.so.3 => /usr/lib32/libthr.so.3 (0x310fb000)
> libc.so.7 => /usr/lib32/libc.so.7 (0x29165000)
> /usr/lib32/libthr.so.3:
> libc.so.7 => /usr/lib32/libc.so.7 (0x29165000)
> [~] # ldconfig -32 -r|grep libcrypto.so.8
> 105:-lcrypto.8 => /usr/local/lib32/compat/system/libcrypto.so.8
> [~] # readelf -a /usr/local/lib32/compat/system/libssl.so|egrep "(path|NEEDED)"
> 0x00000001 (NEEDED) Shared library: [libcrypto.so.8]
> 0x00000001 (NEEDED) Shared library: [libthr.so.3]
> 0x00000001 (NEEDED) Shared library: [libc.so.7]
> 0x0000000f (RPATH) Library rpath: [/usr/local/lib]
My guess is it is the RPATH. I am not sure without a simple debug ability.
--
Regards,
Bryan Drewery
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL: <http://lists.freebsd.org/pipermail/svn-src-head/attachments/20140829/947fc41c/attachment.sig>
More information about the svn-src-head
mailing list