malloc() call somehow calling the rtld malloc() implementaion

Fri Jan 27 23:55:24 UTC 2017

On 27 Jan, Konstantin Belousov wrote:
> On Fri, Jan 27, 2017 at 12:19:16PM -0500, Alexander Kabaev wrote:
>> On Fri, 27 Jan 2017 00:31:30 -0800 (PST)
>> Don Lewis <truckman at FreeBSD.org> wrote:
>> 
>> > I've been attempting to get OpenOffice to build properly in a
>> > clang400-import poudriere jail and have run into a mystery.  The build
>> > procedure creates a c++ executable "idlc", which is used to build
>> > other stuff.  The new operator has been overrriden to call a custom
>> > memory allocator, which I have configured to call the system version
>> > of malloc().
>> > 
>> > At some point idlc crashes because it has allocated a 16 byte
>> > structure and the compiler is using "movaps %xmm0,(%rax)" to
>> > initialize it, which requires 16 byte alignment.  Unfortunately this
>> > structure is only 8 byte aligned, causing a bus error.  This
>> > shouldn't be happening because our system malloc() always seems to do
>> > the proper alignment.  It appears that intead of calling the version
>> > of malloc() in libc, the simple version of malloc() built into rtld
>> > is being called instead.
>> > 
>> > GNU gdb 6.1.1 [FreeBSD]
>> > Copyright 2004 Free Software Foundation, Inc.
>> > GDB is free software, covered by the GNU General Public License, and
>> > you are welcome to change it and/or distribute copies of it under
>> > certain conditions. Type "show copying" to see the conditions.
>> > There is absolutely no warranty for GDB.  Type "show warranty" for
>> > details. This GDB was configured as "amd64-marcel-freebsd"...
>> > (gdb) break main
>> > Breakpoint 1 at 0x43b1f6: file idlcmain.cxx, line 34.
>> > (gdb) run @/tmp/r
>> > Starting
>> > program: /wrkdirs/usr/ports/editors/openoffice-devel/work/aoo-4.2.0/main/solver/420/unxfbsdx.pro/bin/idlc
>> > @/tmp/r [New LWP 101304] [New Thread 802616000 (LWP 101304/idlc)]
>> > [Switching to Thread 802616000 (LWP 101304/idlc)]
>> > 
>> > Breakpoint 1, main (argc=2, argv=0x7fffffffb278) at idlcmain.cxx:34
>> > 34	SAL_IMPLEMENT_MAIN_WITH_ARGS(argc, argv)
>> > (gdb) break malloc
>> > Breakpoint 2 at 0x8006a5f01:
>> > file /var/poudriere/jails/clang400amd64/usr/src/libexec/rtld-elf/malloc.c,
>> > line 163. (gdb) cont Continuing.
>> > Trace 12474/1: "Min Prioriy for policy '2' == '0'
>> > "
>> > Trace 12474/1: "Max Prioriy for policy '2' == '103'
>> > "
>> > /wrkdirs/usr/ports/editors/openoffice-devel/work/aoo-4.2.0/main/solver/420/unxfbsdx.pro/bin/idlc:
>> > compiling 1 source files ...
>> > Compiling: /wrkdirs/usr/ports/editors/openoffice-devel/work/aoo-4.2.0/main/offapi/com/sun/star/i18n/KParseTokens.idl
>> > 
>> > Breakpoint 2, malloc (nbytes=343)
>> >     at /var/poudriere/jails/clang400amd64/usr/src/libexec/rtld-elf/malloc.c:163
>> > 163	/var/poudriere/jails/clang400amd64/usr/src/libexec/rtld-elf/malloc.c:
>> > No such file or directory.
>> > in /var/poudriere/jails/clang400amd64/usr/src/libexec/rtld-elf/malloc.c
>> > Current language:  auto; currently minimal (gdb) cont
>> > Continuing.
>> > 
>> > Breakpoint 2, malloc (nbytes=32)
>> >     at /var/poudriere/jails/clang400amd64/usr/src/libexec/rtld-elf/malloc.c:163
>> > 163
>> > in /var/poudriere/jails/clang400amd64/usr/src/libexec/rtld-elf/malloc.c
>> > (gdb) cont Continuing.
>> > Trace 13112/2: "ChildStatusProc : starting
>> > '/wrkdirs/usr/ports/editors/openoffice-devel/work/aoo-4.2.0/main/solver/420/unxfbsdx.pro/bin/ucpp'" [New
>> > Thread 802616500 (LWP 100249/idlc)] [Switching to Thread 802616500
>> > (LWP 100249/idlc)]
>> > 
>> > Breakpoint 2, malloc (nbytes=19)
>> >     at /var/poudriere/jails/clang400amd64/usr/src/libexec/rtld-elf/malloc.c:163
>> > 163
>> > in /var/poudriere/jails/clang400amd64/usr/src/libexec/rtld-elf/malloc.c
>> > (gdb) cont Continuing.
>> > 
>> > Breakpoint 2, malloc (nbytes=34)
>> >     at /var/poudriere/jails/clang400amd64/usr/src/libexec/rtld-elf/malloc.c:163
>> > 163
>> > in /var/poudriere/jails/clang400amd64/usr/src/libexec/rtld-elf/malloc.c
>> > (gdb) cont Continuing.
>> > 
>> > Breakpoint 2, malloc (nbytes=16)
>> >     at /var/poudriere/jails/clang400amd64/usr/src/libexec/rtld-elf/malloc.c:163
>> > 163
>> > in /var/poudriere/jails/clang400amd64/usr/src/libexec/rtld-elf/malloc.c
>> > (gdb) cont Continuing.
>> > 
>> > Breakpoint 2, malloc (nbytes=16)
>> >     at /var/poudriere/jails/clang400amd64/usr/src/libexec/rtld-elf/malloc.c:163
>> > 163
>> > in /var/poudriere/jails/clang400amd64/usr/src/libexec/rtld-elf/malloc.c
>> > (gdb) cont Continuing.
>> > 
>> > Breakpoint 2, malloc (nbytes=16)
>> >     at /var/poudriere/jails/clang400amd64/usr/src/libexec/rtld-elf/malloc.c:163
>> > 163
>> > in /var/poudriere/jails/clang400amd64/usr/src/libexec/rtld-elf/malloc.c
>> > (gdb) cont Continuing.
>> > 
>> > Breakpoint 2, malloc (nbytes=16)
>> >     at /var/poudriere/jails/clang400amd64/usr/src/libexec/rtld-elf/malloc.c:163
>> > 163
>> > in /var/poudriere/jails/clang400amd64/usr/src/libexec/rtld-elf/malloc.c
>> > (gdb) cont Continuing.
>> > sizeof(AstExprValue)=16
>> > 
>> > Program received signal SIGBUS, Bus error.
>> > [Switching to Thread 802616000 (LWP 101304/idlc)]
>> > 0x0000000000478cc2 in AstExpression::eval_bit_op (this=0x802633dc8, 
>> >     ek=EK_const) at astexpression.cxx:1001
>> > 1001	    std::auto_ptr< AstExprValue	> retval(new
>> > AstExprValue());
>> > 
>> > 
>> > idlc is linked to libc.so:
>> > 
>> > /wrkdirs/usr/ports/editors/openoffice-devel/work/aoo-4.2.0/main/solver/420/unxfbsdx.pro/bin/idlc:
>> > 	libreg.so.3
>> > => /wrkdirs/usr/ports/editors/openoffice-devel/work/aoo-4.2.0/main/solver/420/unxfbsdx.pro/lib/libreg.so.3
>> > (0x8008b9000) libuno_sal.so.3
>> > => /wrkdirs/usr/ports/editors/openoffice-devel/work/aoo-4.2.0/main/solver/420/unxfbsdx.pro/lib/libuno_sal.so.3
>> > (0x800c00000) libuno_salhelpergcc3.so.3
>> > => /wrkdirs/usr/ports/editors/openoffice-devel/work/aoo-4.2.0/main/solver/420/unxfbsdx.pro/lib/libuno_salhelpergcc3.so.3
>> > (0x801040000) libm.so.5 => /lib/libm.so.5 (0x801244000) libc++.so.1
>> > => /usr/lib/libc++.so.1 (0x80146e000) libcxxrt.so.1
>> > => /lib/libcxxrt.so.1 (0x801735000) libgcc_s.so.1
>> > => /lib/libgcc_s.so.1 (0x801953000) libthr.so.3 => /lib/libthr.so.3
>> > (0x801b69000) libc.so.7 => /lib/libc.so.7 (0x801d91000) libstore.so.3
>> > => /wrkdirs/usr/ports/editors/openoffice-devel/work/aoo-4.2.0/main/solver/420/unxfbsdx.pro/lib/libstore.so.3
>> > (0x802152000)
>> > 
>> > 
>> > If I create a simple test program that calls malloc() and set a
>> > breakpoint in malloc(), the breakpoint gets set in the rtld version,
>> > but the the libc version of malloc is what gets called.
>> > 
>> > What the heck is going on here, and how can I fix it?
>> > 
>> 
>> rtld on my system does not have malloc exposed as dynamic symbol, it
>> cannot possibly be used for symbol resolution by any outside module.
> 
> Sure.
> 
> Also, the fact that rtld internal malloc is called does not mean that it
> is called by the application. There is no backtrace which would describe
> the reason for rtld allocating memory in reported cases. Generally,
> rtld needs memory for TLS allocation (the program is definitely linked
> with libthr, and even libc uses TLS), and, of course, to create data
> structures to track dlopen-ed objects.

Good point.  Unfortunately I didn't have ports gdb available to look for
calls for libc malloc().