Strange rtld-elf failure on stable/12 [Was: libcrypto.so.111 linked binaries SIGSEGV (in bhyve guest)]

[Harry Schmalzbauer]

Am 20.02.2019 um 17:51 schrieb Harry Schmalzbauer:
> Hello,
>
…
> gdb shows:
> Core was generated by `/usr/sbin/auditdistd'.
> Program terminated with signal 11, Segmentation fault.
> Reading symbols from /lib/libutil.so.9...Reading symbols from 
> /usr/lib/debug//lib/libutil.so.9.debug...done.
> done.
> Loaded symbols for /lib/libutil.so.9
> Reading symbols from /libexec/ld-elf.so.1...Reading symbols from 
> /usr/lib/debug//libexec/ld-elf.so.1.debug...done.
> done.
> Loaded symbols for /libexec/ld-elf.so.1
> #0  memset (dest=0x80056f790, c=0, len=<value optimized out>)
>     at 
> /usr/local/share/deploy-tools/RELENG_12/src/libexec/rtld-elf/rtld.c:5624
> 5624                    ((char *)dest)[i] = c;
> (gdb) bt
> #0  memset (dest=0x80056f790, c=0, len=<value optimized out>)
>     at 
> /usr/local/share/deploy-tools/RELENG_12/src/libexec/rtld-elf/rtld.c:5624
> #1  0x0000000800235b07 in map_object (fd=3, path=0x800246140 
> "/lib/libcrypto.so.111",
>     sb=0x7fffffffd4a8)
>     at 
> /usr/local/share/deploy-tools/RELENG_12/src/libexec/rtld-elf/map_object.c:249
> #2  0x0000000800230806 in load_object (name=0x201dba 
> "libcrypto.so.111", fd_u=-1,
>     refobj=0x800248000, flags=<value optimized out>)
>     at 
> /usr/local/share/deploy-tools/RELENG_12/src/libexec/rtld-elf/rtld.c:2493
> #3  0x0000000800229972 in _rtld (sp=<value optimized out>, 
> exit_proc=0x7fffffffea30,
>     objp=0x7fffffffea38)
>     at 
> /usr/local/share/deploy-tools/RELENG_12/src/libexec/rtld-elf/rtld.c:2315
> #4  0x0000000800228019 in .rtld_start ()
>     at 
> /usr/local/share/deploy-tools/RELENG_12/src/libexec/rtld-elf/amd64/rtld_start.S:39
> #5  0x0000000000000000 in ?? ()
> Current language:  auto; currently minimal
>
> Any help highly appreciated.
>
> This is with a live CD (amd64), compiled with stable/12 from today (so 
> clang 7.01).
> The bhyve guest has 2GB hardwired and ran stable/11 beforehand, which 
> compiled the live CD.
> bhyve host is 11.2.  But that shouldn't play a role, does it?

I'm really interested what happens here.
I built stable/11 in that bhyve guest and updated that guest to 
stable/11 from yesterday.
To my surpise llvm 7.01 was also merged to stable/11.  Thank you for 
that great supprt!
No problems with any binary in the stable/11 bhyve guest.

Then I built stable/12 in that re-built stable/11 guest.
As result, again all binaries linked to /lib/libcrypto.so.111 crash 
(signal 11) with the stable/12 iso in the same bhyve guest.

Here the example from ntpq:
Program terminated with signal 11, Segmentation fault.
Reading symbols from /lib/libedit.so.7...Reading symbols from 
/usr/lib/debug//lib/libedit.so.7.debug...done.
done.
Loaded symbols for /lib/libedit.so.7
Reading symbols from /lib/libm.so.5...Reading symbols from 
/usr/lib/debug//lib/libm.so.5.debug...done.
done.
Loaded symbols for /lib/libm.so.5
Reading symbols from /libexec/ld-elf.so.1...Reading symbols from 
/usr/lib/debug//libexec/ld-elf.so.1.debug...done.
done.
#0  memset (dest=0x8005ef790, c=0, len=<value optimized out>) at 
/usr/local/share/deploy-tools/RELENG_12/src/libexec/rtld-elf/rtld.c:5624
5624                    ((char *)dest)[i] = c;
(gdb) bt
#0  memset (dest=0x8005ef790, c=0, len=<value optimized out>) at 
/usr/local/share/deploy-tools/RELENG_12/src/libexec/rtld-elf/rtld.c:5624
#1  0x000000080025db07 in map_object (fd=3, path=0x80026e1a0 
"/lib/libcrypto.so.111", sb=0x7fffffffd4c8) at 
/usr/local/share/deploy-tools/RELENG_12/src/libexec/rtld-elf/map_object.c:249
#2  0x0000000800258806 in load_object (name=0x201b40 "libcrypto.so.111", 
fd_u=-1, refobj=0x800270000, flags=<value optimized out>) at 
/usr/local/share/deploy-tools/RELENG_12/src/libexec/rtld-elf/rtld.c:2493
#3  0x0000000800251972 in _rtld (sp=<value optimized out>, 
exit_proc=0x7fffffffea50, objp=0x7fffffffea58) at 
/usr/local/share/deploy-tools/RELENG_12/src/libexec/rtld-elf/rtld.c:2315
#4  0x0000000800250019 in .rtld_start () at 
/usr/local/share/deploy-tools/RELENG_12/src/libexec/rtld-elf/amd64/rtld_start.S:39
#5  0x0000000000000000 in ?? ()

So please correct me if I'm comletely wrong, but the problem here seems 
to be reproducably rtld-elf related.
Unfortunately I don't know anything about object files and linkers and 
the related fundamental stuff.
But maybe someone else has an idea what's going wrong here?

Thanks,

-Harry