linked ssl libraries to binary

[Jeremy Chadwick]

On Tue, Mar 04, 2008 at 12:05:22PM +0000, Chris wrote:
> I have a freebsd 6.3 server and a freebsd 7.0 server, I have a binary
> I run of the freebsd 7 box but has recently been crashing, the binary
> in question is ezbounce.
> 
> It was previously running for weeks no problems at all and then during
> the past 2 weeks or so its had problems.
> 
> Like many shell programs it has a configure script and you then run
> make or gmake to compile the binary.

You know there's /usr/ports/irc/ezbounce, right?  Well, I suppose that
only applies if you actually maintain/run the servers in question.  But
apparently you do (see below).

> So I then decided to move the binary I compiled on freebsd 6 over to
> the freebsd 7 box and when I ran ldd on the binary to my surprise it
> is using the base libraries on freebsd 7.

This doesn't come as much of a surprise.  The binary actually references
libraries by names such as libXXX.so, not libXXX.so.NUMBER.

> ldd on binary on freebsd 6
> 
>         libssl.so.5 => /usr/local/lib/libssl.so.5 (0x48102000)
>         libcrypto.so.5 => /usr/local/lib/libcrypto.so.5 (0x48143000)
>         libcrypt.so.3 => /lib/libcrypt.so.3 (0x4829f000)
>         libboost_iostreams.so => /usr/local/lib/libboost_iostreams.so (0x482b8000)
>         libstdc++.so.5 => /usr/lib/libstdc++.so.5 (0x482c0000)
>         libm.so.4 => /lib/libm.so.4 (0x48396000)
>         libpthread.so.2 => /lib/libpthread.so.2 (0x483ac000)
>         libc.so.6 => /lib/libc.so.6 (0x483d3000)
>         libbz2.so.2 => /usr/lib/libbz2.so.2 (0x484cb000)
>         libz.so.3 => /lib/libz.so.3 (0x484dc000)
>
> ldd on same binary on freebsd 7
> 
>         libssl.so.5 => /usr/lib/libssl.so.5 (0x28101000)
>         libcrypto.so.5 => /lib/libcrypto.so.5 (0x28142000)

The libssl.so being used by the ezbounce binary you have, on RELENG_7,
is using the base system's version.  This is NOT compatible, AFAIK, as
the libssl.so on RELENG_6.

The same issue applies to libcrypto.so.

>         libcrypt.so.3 => /usr/local/lib/compat/libcrypt.so.3 (0x2829a000)
>         libboost_iostreams.so => /usr/local/lib/libboost_iostreams.so (0x282b2000)
>         libstdc++.so.5 => /usr/local/lib/compat/libstdc++.so.5 (0x282bd000)
>         libm.so.4 => /usr/local/lib/compat/libm.so.4 (0x28388000)
>         libpthread.so.2 => /usr/local/lib/compat/libpthread.so.2 (0x2839e000)
>         libc.so.6 => /usr/local/lib/compat/libc.so.6 (0x283c3000)
>         libc.so.7 => /lib/libc.so.7 (0x284a9000)
>         libbz2.so.3 => /usr/lib/libbz2.so.3 (0x285a4000)
>         libz.so.4 => /lib/libz.so.4 (0x285b4000)
>         libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x285c6000)
>         libm.so.5 => /lib/libm.so.5 (0x286ba000)
>         libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x286cf000)
>         libthr.so.3 => /lib/libthr.so.3 (0x286da000)

I can't explain what's with the "double-linking" of some libraries, e.g.
two linked in-versions of libc, libm, libstdc++, and others.  It's
possible this is "normal", but it seems like it would cause a crash.

I do know that FreeBSD has some sort of internal version numbering for
symbols in shared libraries, but I don't know if it was introduced in
RELENG_7, or if RELENG_6 had it.  If it's new as of RELENG_7, then I can
see how a binary built on RELENG_6 _might_ call the wrong "version" of a
function.  nm(1) output would be able to help with this, I think.

> The binary doesnt run on the freebsd 7 either it coredumps even tho I
> have compat6x installed, typically in the past I have had no problems
> at all using binaries compiled on old freebsd versions on newer
> versions I just had to install the appropriate compat package.

I would strongly recommend against relying on compat6x for anything that
can be recompiled.  I have never trusted the compat libraries, because I
don't like to play risky business with multiple versions of a shared
library on a machine (see below).

You did not provide a crash dump or gdb output of the program, so it's
hard to say where the actual crash (SSL, libc, libm, etc.) occurred.
But then again, is it worth debugging when.....

> Here is the ldd when compiled on the freebsd 7 box

.....you can recompile it?  :-) You should be doing this anyways.  So
what's the problem -- or is this more of a "I'm curious how ld.so works"
inquiry?

> Is the output for the ssl libraries skewed because both the local
> filenames and the base filenames are the same? as there is a
> /usr/local/lib/libssl.so.5 and a /usr/lib/libssl.so.5.  Or does this
> mean that it really is linked against the base libraries as I am
> wondering how that is possible when the same binary is linked against
> different libraries on a different machine.

I indirectly answered this in my 2nd paragraph.  Welcome to the UNIX
equivalent of "DLL Hell" on Windows -- and why you should *always*
recompile programs when the major version of a shared library (.so)
changes.  I cannot stress this enough.

All that said: you might be able to get around the problem in question
by setting LD_LIBRARY_PATH="/usr/local/lib/compat" when running the
RELENG_6-compiled ezbounce binary on RELENG_7, e.g.:

  sh# LD_LIBRARY_PATH="/usr/local/lib/compat" ezbounce

You could also do (with possibly mixed results, since ldd is linked to
libc.so, so this might cause /usr/bin/ldd to try and use the compat6x
version of libc.so):

  sh# LD_LIBRARY_PATH="/usr/local/lib/compat" ldd ezbounce

-- 
| Jeremy Chadwick                                    jdc at parodius.com |
| Parodius Networking                           http://www.parodius.com/ |
| UNIX Systems Administrator                      Mountain View, CA, USA |
| Making life hard for others since 1977.                  PGP: 4BD6C0CB |