VirtualBox (sometimes) not starting/hanging after recent openssl updates

[Ivo Karabojkov]

Hi,

I've explained my tests already to the thread you mention. Here is my
ldd output from a working system:
# ldd /usr/local/lib/virtualbox/VBoxRT.so
/usr/local/lib/virtualbox/VBoxRT.so:
        librt.so.1 => /usr/lib/librt.so.1 (0x801816000)
        libz.so.6 => /lib/libz.so.6 (0x801a1c000)
        libthr.so.3 => /lib/libthr.so.3 (0x801c32000)
        libxml2.so.2 => /usr/local/lib/libxml2.so.2 (0x801e57000)
        libcurl.so.4 => /usr/local/lib/libcurl.so.4 (0x8021ea000)
        libssl.so.7 => /usr/lib/libssl.so.7 (0x80243d000)
        libcrypto.so.7 => /lib/libcrypto.so.7 (0x8026a8000)
        libc++.so.1 => /usr/lib/libc++.so.1 (0x802a9c000)
        libcxxrt.so.1 => /lib/libcxxrt.so.1 (0x802d5c000)
        libm.so.5 => /lib/libm.so.5 (0x802f78000)
        libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x8031a0000)
        libc.so.7 => /lib/libc.so.7 (0x80081f000)
        liblzma.so.5 => /usr/lib/liblzma.so.5 (0x8033ae000)
        libgssapi.so.10 => /usr/lib/libgssapi.so.10 (0x8035d3000)
        libgssapi_krb5.so.10 => /usr/lib/libgssapi_krb5.so.10 (0x8037dc000)
        libheimntlm.so.11 => /usr/lib/libheimntlm.so.11 (0x8039fa000)
        libkrb5.so.11 => /usr/lib/libkrb5.so.11 (0x803c00000)
        libhx509.so.11 => /usr/lib/libhx509.so.11 (0x803e78000)
        libcom_err.so.5 => /usr/lib/libcom_err.so.5 (0x8040c2000)
        libasn1.so.11 => /usr/lib/libasn1.so.11 (0x8042c4000)
        libwind.so.11 => /usr/lib/libwind.so.11 (0x804561000)
        libheimbase.so.11 => /usr/lib/libheimbase.so.11 (0x804789000)
        libroken.so.11 => /usr/lib/libroken.so.11 (0x80498d000)
        libcrypt.so.5 => /lib/libcrypt.so.5 (0x804b9f000)
        libheimipcc.so.11 => /usr/lib/private/libheimipcc.so.11
(0x804dbf000)

I have no experience in debugging yet but I'd be glad to help.


On 29.3.2015 г. 21:27 ч., Christoph Moench-Tegeder wrote:
> Hi,
>
> for reference, prior reports (not mine):
> https://lists.freebsd.org/pipermail/freebsd-emulation/2015-March/012381.html
> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=198952
>
> I got bitten by the very same problem - "NS_ERROR_FACTORY_NOT_REGISTERED"
> when running VirtualBox "as usual", "VERR_INVALID_POINTER" and "environment
> corrupt" when using a DEBUG-enabled VirtualBox.
>
> As I was pretty sure I had a working VirtualBox 4.3.26 before, I tracked
> all the changes since the last successful VirtualBox use, and the only
> "significant" thing was that I upgraded my base system for
> FreeBSD-SA-15:06.openssl (and the fix for that, so I'm up-to-date with
> releng/10.1 (r280275).
>
> After some time of poking the source and adding more debug statements
> and asserts (debugging suid code...), I got the following backtrace in
> my core dump (abbreviated):
> (gdb) bt
> #0  0x00000008033257c5 in OPENSSL_ia32_cpuid ()
>    from /usr/local/lib/libcrypto.so.8
> #1  0x0000000805f88eb9 in ?? () from /lib/libcrypto.so.7
> #2  0x0000000805e8f84e in _init () from /lib/libcrypto.so.7
> #3  0x00007fffffffc760 in ?? ()
> #4  0x000000080060e6bf in ?? () from /libexec/ld-elf.so.1
> #5  0x0000000800612d87 in ?? () from /libexec/ld-elf.so.1
> #6  0x000000080060fad3 in ?? () from /libexec/ld-elf.so.1
> #7  0x00000000004042d9 in supR3HardenedMainInitRuntime (fFlags=3)
>
> Having different versions of a library calling each other (base openssl
> vs. ports openssl) is most certainly a bad thing.
> More digging showed that the libcryptos where pulled in by libcurl.so.4.
> That (curl) in turn links against ports openssl when found (there's
> even code in the Makefile to stop the build when one tries forcing the
> use of base openssl), but happily pulls in base openssl via it's
> dependencies. One instance of the base openssl was found in librtmp
> and could be solved by recompiling that. The other problem was with
> the use of "GSSAPI_BASE" (base system GSSAPI) in curl - which is the
> default, according to a curl-less machine in the corner.
>
> In the short run, people affected by the VirtualBox problem should
> check if /usr/local/lib/virtualbox/VBoxRT.so links against both
> (base and ports) libcryptos (use ldd) - if so, check libcurl (and if
> that does not help, the other shared libraries) for libcrypto mixup.
>
> In the slightly longer run - how do we want to fix this? I can imagine
> preventing the use of GSSAPI_BASE when ports openssl is detected - there's
> similar code already for preventing WITH_OPENSSL_BASE usage when
> ports openssl is installed. OTOH this will not help in case the "wrong"
> openssl is pulled in via some other dependency (librtmp in my case).
>
> Regards,
> Christoph
>


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://lists.freebsd.org/pipermail/freebsd-emulation/attachments/20150330/fd3e4cda/attachment.sig>