svn commit: r324938 - head/contrib/jemalloc/include/jemalloc/internal

[Michal Meloun]

On Fri, Oct 27, 2017 at 19:48:43 +0200, Michal Meloun wrote:
> On 27.10.2017 17:21, Brooks Davis wrote:
>> On Fri, Oct 27, 2017 at 06:08:41PM +0300, Konstantin Belousov
>> wrote:
>>> On Fri, Oct 27, 2017 at 02:53:26PM +0200, Michal Meloun wrote:
>>>> Sorry for top posting That's pity, we have clear problem in
>>>> rtld code :( See: 
>>>> ----------------------------------------------------- RESCUE
>>>> WITHOUT JEMALLOC_ALIGNED(16); 
>>>> ----------------------------------------------------- Program
>>>> Headers: TLS            0xa732b0 0x00a832b0 0x00a832b0 0x00b40
>>>> 0x011bc R   0x8 Section Headers: 04     .tdata .tbss
>>>> .init_array .fini_array .jcr .got Dump: 00a832b0
>>>> <__je_tsd_tls+0xa832b0>: a832b0:       00000005
>>>> 
>>>> GDB (gdb) b tsd_fetch_impl Breakpoint 1 at 0x7c4c08:
>>>> tsd_fetch_impl. (6 locations) (gdb) r Starting program:
>>>> /usr/src/rescue.noalign sh
>>>> 
>>>> Breakpoint 1, tsd_fetch_impl (init=true, minimal=false) at 
>>>> /usr/src/contrib/jemalloc/include/jemalloc/internal/tsd.h:261 
>>>> 261             tsd_t *tsd = tsd_get(init); (gdb) n 263
>>>> if (!init && tsd_get_allocates() && tsd == NULL) {
>>>> 
>>>> (gdb) p tsd $1 = (tsd_t *) 0x20c83008
>>>> 
>>>> (gdb) p *tsd $2 = {state = 5 '\005', ....
>>>> 
>>>> (gdb) p *((tsd_t *)0x00a832b0) $3 = {state = 5 '\005', ...
>>>> 
>>>> 
>>>> 
>>>> ----------------------------------------------------- RESCUE
>>>> WITH JEMALLOC_ALIGNED(16); 
>>>> ----------------------------------------------------- Program
>>>> Headers: TLS            0xa732b0 0x00a832b0 0x00a832b0 0x00b40
>>>> 0x011bc R   0x10 Section Headers: 04     .tdata .tbss
>>>> .init_array .fini_array .jcr .got Dump: 00a832b0
>>>> <__je_tsd_tls+0xa832b0>: a832b0:       00000005
>>>> 
>>>> GDB (gdb) b tsd_fetch_impl Breakpoint 1 at 0x7c4c08:
>>>> tsd_fetch_impl. (6 locations) (gdb) r Starting program:
>>>> /usr/obj/usr/src/rescue/rescue/rescue sh Breakpoint 1,
>>>> tsd_fetch_impl (init=true, minimal=false) at 
>>>> /usr/src/contrib/jemalloc/include/jemalloc/internal/tsd.h:261 
>>>> 261             tsd_t *tsd = tsd_get(init); (gdb) n 263
>>>> if (!init && tsd_get_allocates() && tsd == NULL) {
>>>> 
>>>> (gdb) p tsd $1 = (tsd_t *) 0x20c83010
>>>> 
>>>> (gdb) p *tsd $2 = {state = 0 '\000', ...
>>>> 
>>>> (gdb) p *((tsd_t *)0x00a832b0) $3 = {state = 5 '\005', ...
>>>> 
>>>> !!!! BUT p *(tsd - 8 bytes) !!!!!!!!!! (gdb) p *((tsd_t
>>>> *)0x20c83008) $4 = {state = 5 '\005', ...
>>>> 
>>>> ----------------------------------------------------- So it's
>>>> clear that:
>>>> 
>>>> - both binaries are valid, .tdata section have valid data. -
>>>> requested alignment is propagated to binary. - .tdata section
>>>> is properly loaded to memory because p *((tsd_t *)0x00a832b0)
>>>> is {state = 5 '\005' in both cases
>>>> 
>>>> - a per thread copy of .tdata respect requested alignment but
>>>> the original data was copied to  unaligned address. because for
>>>> aligned binary p *tsd is {state = 0 '\000', ... p *(tsd - 8
>>>> bytes) is {state = 5 '\005'
>>>> 
>>>> I'm right? Kib, please, can you help us?
>>> 
>>> Does it happen for rescue binary ?
>>> 
>>> Note that the binary is linked static, so the problem is in
>>> lib/libc/gen/tls.c and not in rtld.  There, I do not see any real
>>> use of the phdr' p_align value.
>>> 
> Ahh, right, good catch.
>>> BTW, is rescue linked to libthr ?
> Imho not.
My bad, rescue *is* linked with libthr. And after looking to
libthr/arch/arm/pthread_md.h, I'm confused much much more...


>> 
>> There isn't alignment support for TLS in static binaries.  I've
>> fixed this in CheriBSD and am planning to upstream the fixes at
>> some point. The fix for variant I is in:
>> 
>> https://github.com/CTSRD-CHERI/cheribsd/commit/3cfb124ebb9fdb545dad8436a04dd58c05b33f4b
>>
>> 
> The real alignment have no effect for program itself, it's important
> only for AddressSanitizer.
> 
> But there is something what's still missing me.
> The rescue binary is statically linked, so whole program have only one
> TLS section and full TLS layout is determined by linker.
> The problematic structure,  tsd_tls, is first one in TLS section group:
> 
> readelf -a /usr/obj/usr/src/rescue/rescue/rescue | grep TLS
> 518961: 0000000000000000  2880 TLS     GLOBAL DEFAULT    9 __je_tsd_tls
> and
> 
> objdump -d -j .tdata /usr/obj/usr/src/rescue/rescue/rescue
> 00a832b0 <__je_tsd_tls+0xa832b0>:
> a832b0:       00000005
> 
> but disassembled code expect it at TLS ptr + 0x8 (for code without
> JEMALLOC_ALIGNED) or at TLS ptr + 0x10 (for code with JEMALLOC_ALIGNED)
> 
> [without JEMALLOC_ALIGNED]
>   7c4d90:       e59f0014        ldr     r0, [pc, #20]   ; 7c4dac
> <tsd_get+0x80>
>   7c4d94:       e58d0004        str     r0, [sp, #4]
>   7c4d98:       eb031d14        bl      88c1f0 <__aeabi_read_tp>
>   7c4d9c:       e59dc004        ldr     ip, [sp, #4]
>   7c4da0:       e080000c        add     r0, r0, ip
>   7c4da4:       e1a0d00b        mov     sp, fp
>   7c4da8:       e8bd8800        pop     {fp, pc}
>   7c4dac:       00000008        .word   0x00000008  [0x10 for code
> with  JEMALLOC_ALIGNED]
> 
> so someone other allocates at least one byte from start of TLS and
> linker is aware about this. But again, .data is first member of TLS
> and tsd_tls is first of .tdata and the 0x5 is value of first member of
> tsd_tls structure (.state = tsd_state_uninitialized).
> 
> So who consumes the first bytes in TLS ????
> 
> Michal