svn commit: r339876 - head/libexec/rtld-elf
Mark Millard
marklmi26-fbsd at yahoo.com
Fri Nov 2 07:26:39 UTC 2018
[The backtrace confirms what I reported to Alexander Richardson
and Shawn Webb earlier.]
On 2018-Nov-1, at 6:40 PM, Mark Millard <marklmi26-fbsd at yahoo.com> wrote:
> On 2018-Nov-1, at 5:41 PM, Konstantin Belousov <kostikbel at gmail.com> wrote:
>
>> On Tue, Oct 30, 2018 at 12:45:13PM -0700, Mark Millard wrote:
>>> Konstantin Belousov kostikbel at gmail.com wrote on
>>> Tue Oct 30 18:04:04 UTC 2018 :
>>>
>>>> On Tue, Oct 30, 2018 at 03:32:40PM +0000, Alexander Richardson wrote:
>>>>> On Tue, 30 Oct 2018 at 10:17, Michael Tuexen
>>>>> <Michael.Tuexen at macmic.franken.de> wrote:
>>>>>>
>>>>>>> On 29. Oct 2018, at 22:08, Alex Richardson <arichardson at FreeBSD.org> wrote:
>>>>>>>
>>>>>>> Author: arichardson
>>>>>>> Date: Mon Oct 29 21:08:02 2018
>>>>>>> New Revision: 339876
>>>>>>> URL: https://svnweb.freebsd.org/changeset/base/339876
>>>>>>>
>>>>>>> Log:
>>>>>>> rtld: set obj->textsize correctly
>>>>>>>
>>>>>>> With lld-generated binaries the first PT_LOAD will usually be a read-only
>>>>>>> segment unless you pass --no-rosegment. For those binaries the textsize is
>>>>>>> determined by the next PT_LOAD. To allow both LLD and bfd 2.17 binaries to
>>>>>>> be parsed correctly use the end of the last PT_LOAD that is marked as
>>>>>>> executable instead.
>>>>>>>
>>>>>>> I noticed that the value was wrong while adding some debug prints for some rtld
>>>>>>> changes for CHERI binaries. `obj->textsize` only seems to be used by PPC so the
>>>>>>> effect is untested. However, the value before was definitely wrong and the new
>>>>>>> result matches the phdrs.
>>>>>> I build kernel and world with a revision later than this on a PPC. Buildword
>>>>>> ends up with a world where almost all binaries are segfaulting.... Especially gdb
>>>>>> (but svn, ls or so all segfault).
>>>>>>
>>>>>> Best regards
>>>>>> Michael
>>>>>
>>>>> This is rather surprising since if anything the range of the icache
>>>>> flush should increase rather than decrease after this change.
>>>>>
>>>>> I can only see this causing a behaviour change if we actually need to
>>>>> flush more than just the executable segments.
>>>>> Is it possible that some binary/library contains a non-executable
>>>>> segment as the first PT_LOAD?
>>>>> Or is there some linker script that adds custom PHDRS?
>>>>>
>>>> Could it be that there is a hole between start of the object mapping and
>>>> the last PT_LOADable segment eligible for execution ?
>>>
>>> [This note may be easier to deal with than the first
>>> note that I sent out.]
>>>
>>> [My examples are from devel/powerpc64-xtoolchain-gcc used
>>> to buildworld buildkernel targeting a head -r339076 based
>>> powerpc64 environment. I do that on powerpc64 as well.]
>>>
>>> powerpc64 loads the readonly data and the readonly code in one PT_LOAD,
>>> the first. The 2nd PT_LOAD is for sections without the readonly status,
>>> that includes .got and .plt being spanned. See below from
>>> objdump -ph for /bin/ls :
>>>
>>> Program Header:
>>> PHDR off 0x0000000000000040 vaddr 0x0000000010000040 paddr 0x0000000010000040 align 2**3
>>> filesz 0x0000000000000188 memsz 0x0000000000000188 flags r--
>>> INTERP off 0x00000000000001c8 vaddr 0x00000000100001c8 paddr 0x00000000100001c8 align 2**0
>>> filesz 0x0000000000000015 memsz 0x0000000000000015 flags r--
>>> LOAD off 0x0000000000000000 vaddr 0x0000000010000000 paddr 0x0000000010000000 align 2**16
>>> filesz 0x000000000000910c memsz 0x000000000000910c flags r-x
>>> LOAD off 0x0000000000009110 vaddr 0x0000000010019110 paddr 0x0000000010019110 align 2**16
>>> filesz 0x0000000000000ee0 memsz 0x00000000000010e8 flags rw-
>>> DYNAMIC off 0x0000000000009138 vaddr 0x0000000010019138 paddr 0x0000000010019138 align 2**3
>>> filesz 0x00000000000001c0 memsz 0x00000000000001c0 flags rw-
>>> NOTE off 0x00000000000001e0 vaddr 0x00000000100001e0 paddr 0x00000000100001e0 align 2**2
>>> filesz 0x0000000000000030 memsz 0x0000000000000030 flags r--
>>> STACK off 0x0000000000000000 vaddr 0x0000000000000000 paddr 0x0000000000000000 align 2**4
>>> filesz 0x0000000000000000 memsz 0x0000000000000000 flags rw-
>>
>> We only need program headers, and we only need them from the object
>> which load causes the fault. It might be not the binary but some library
>> that triggers the fault. So the backtrace and some information from the
>> state of the image is needed.
>>
>> You can build only rtld and use it as the standalone program to initiate
>> the image activation:
>> <path>/ld-elf.so.1 /bin/ls
>> or similar.
>
> . . .
It stops when the dcbst in __syncicache runs into an address in
the p_align 65536 caused hole between the two PT_LOAD's with PF_X.
/bin/ls itself has such a hole, as do the .so libraries involved.
Details follow.
(gdb) run /bin/ls
Starting program: /usr/obj/powerpcvtsc_gcc421/powerpc.powerpc/usr/src/powerpc.powerpc/libexec/rtld-elf/ld-elf.so.1.full /bin/ls
Program received signal SIGSEGV, Segmentation fault.
__syncicache (from=0x1800000, len=102400) at /usr/src/lib/libc/powerpc/gen/syncicache.c:94
94 __asm __volatile ("dcbst 0,%0" :: "r"(p));
(gdb) bt
#0 __syncicache (from=0x1800000, len=102400) at /usr/src/lib/libc/powerpc/gen/syncicache.c:94
#1 0x01012b58 in reloc_non_plt (obj=0x41041000, obj_rtld=<optimized out>, flags=4, lockstate=0x0) at /usr/src/libexec/rtld-elf/powerpc/reloc.c:330
#2 0x010175f4 in relocate_object (obj=0x41041000, bind_now=0 '\000', rtldobj=0x103ea88 <obj_rtld>, flags=4, lockstate=0x0) at /usr/src/libexec/rtld-elf/rtld.c:2849
#3 0x0101776c in relocate_objects (first=<optimized out>, bind_now=0 '\000', rtldobj=0x103ea88 <obj_rtld>, flags=4, lockstate=0x0) at /usr/src/libexec/rtld-elf/rtld.c:2908
#4 0x0101beb0 in _rtld (sp=<optimized out>, exit_proc=0xffffdc08, objp=0xffffdc0c) at /usr/src/libexec/rtld-elf/rtld.c:677
#5 0x0101217c in .rtld_start () at /usr/src/libexec/rtld-elf/powerpc/rtld_start.S:98
Backtrace stopped: frame did not save the PC
This is because there are 2 PT_LOAD's that have PF_X but
for which there is an address range between that spans
may pages for which there is no header entry for, in part
a consequence of p_align being 65536 so forcing lots of
space between entries 2 and 3 ( /bin/ls example):
entry: 2
p_type: PT_LOAD
p_offset: 0
p_vaddr: 0x1800000
p_paddr: 0x1800000
p_filesz: 34112
p_memsz: 34112
p_flags: PF_X|PF_R
p_align: 65536
entry: 3
p_type: PT_LOAD
p_offset: 34112
p_vaddr: 0x1818540
p_paddr: 0x1818540
p_filesz: 316
p_memsz: 1752
p_flags: PF_X|PF_W|PF_R
p_align: 65536
( check 2's p_paddr+ p_memsz vs. 3's p_vaddr
Later below the failing address will be shown
as between those. )
[powerpc64 only gets one PT_LOAD with PF_X and
so can not have space between such PT_LOAD's
as things currently are.]
The source code related details are as follows
and shows that it was processing /bin/ls itself
( obj_main->path = 0x41042000 "/bin/ls" and
the failing address ) :
[The presentation order is not necessarily the same
as my exploration order.]
(gdb) list
89 off = (u_int)from & (cacheline_size - 1);
90 l = len += off;
91 p = (char *)from - off;
92
93 do {
94 __asm __volatile ("dcbst 0,%0" :: "r"(p));
95 p += cacheline_size;
96 } while ((l -= cacheline_size) > 0);
97 __asm __volatile ("sync");
98 p = (char *)from - off;
[Reminder for entry 2:
p_vaddr: 0x1800000
p_memsz: 34112
0x1800000+34112 = 0x1808540
]
(gdb) print p
$4 = 0x1809000 <error: Cannot access memory at address 0x1809000>
(gdb) up
#1 0x01012b58 in reloc_non_plt (obj=0x41041000, obj_rtld=<optimized out>, flags=4, lockstate=0x0) at /usr/src/libexec/rtld-elf/powerpc/reloc.c:330
330 __syncicache(obj->mapbase, obj->textsize);
(gdb) list
325 done:
326 if (cache != NULL)
327 free(cache);
328
329 /* Synchronize icache for text seg in case we made any changes */
330 __syncicache(obj->mapbase, obj->textsize);
331
332 return (r);
333 }
334
(gdb) up
#2 0x010175f4 in relocate_object (obj=0x41041000, bind_now=0 '\000', rtldobj=0x103ea88 <obj_rtld>, flags=4, lockstate=0x0) at /usr/src/libexec/rtld-elf/rtld.c:2849
2849 if (reloc_non_plt(obj, rtldobj, flags, lockstate))
(gdb) list
2844 /* There are relocations to the write-protected text segment. */
2845 if (obj->textrel && reloc_textrel_prot(obj, true) != 0)
2846 return (-1);
2847
2848 /* Process the non-PLT non-IFUNC relocations. */
2849 if (reloc_non_plt(obj, rtldobj, flags, lockstate))
2850 return (-1);
2851
2852 /* Re-protected the text segment. */
2853 if (obj->textrel && reloc_textrel_prot(obj, false) != 0)
(gdb) up
#3 0x0101776c in relocate_objects (first=<optimized out>, bind_now=0 '\000', rtldobj=0x103ea88 <obj_rtld>, flags=4, lockstate=0x0) at /usr/src/libexec/rtld-elf/rtld.c:2908
2908 error = relocate_object(obj, bind_now, rtldobj, flags,
(gdb) list
2903
2904 for (error = 0, obj = first; obj != NULL;
2905 obj = TAILQ_NEXT(obj, next)) {
2906 if (obj->marker)
2907 continue;
2908 error = relocate_object(obj, bind_now, rtldobj, flags,
2909 lockstate);
2910 if (error == -1)
2911 break;
2912 }
(gdb) up
#4 0x0101beb0 in _rtld (sp=<optimized out>, exit_proc=0xffffdc08, objp=0xffffdc0c) at /usr/src/libexec/rtld-elf/rtld.c:677
677 if (relocate_objects(obj_main,
(gdb) list
672 * block even if they didn't ask for it.
673 */
674 allocate_tls_offset(entry->obj);
675 }
676
677 if (relocate_objects(obj_main,
678 ld_bind_now != NULL && *ld_bind_now != '\0',
679 &obj_rtld, SYMLOOK_EARLY, NULL) == -1)
680 rtld_die();
681
(gdb) up
#5 0x0101217c in .rtld_start () at /usr/src/libexec/rtld-elf/powerpc/rtld_start.S:98
98 bl _rtld at plt /* &_start = _rtld(sp, &exit_proc, &obj_main)*/
(gdb) list
93 addi %r3,%r4,-4 /* locate argc ptr, &argv[-1] */
94
95 addi %r4,%r1,8 /* &exit_proc on stack */
96 addi %r5,%r1,12 /* &obj_main on stack */
97
98 bl _rtld at plt /* &_start = _rtld(sp, &exit_proc, &obj_main)*/
99 mtlr %r3
100
101 /*
102 * Restore args, with new obj/exit proc
(gdb) down
#4 0x0101beb0 in _rtld (sp=<optimized out>, exit_proc=0xffffdc08, objp=0xffffdc0c) at /usr/src/libexec/rtld-elf/rtld.c:677
677 if (relocate_objects(obj_main,
. . .
(gdb) print *obj_main
$3 = {magic = 0, version = 0, next = {tqe_next = 0x41041200, tqe_prev = 0x103ea68 <obj_list>}, path = 0x41042000 "/bin/ls", origin_path = 0x0, refcount = 1, holdcount = 0, dl_refcount = 0,
mapbase = 0x1800000 "\177ELF\001\002\001\t", mapsize = 102400, textsize = 102400, vaddrbase = 25165824, relocbase = 0x0, dynamic = 0x1818554,
entry = 0x18022e0 "|\b\002\246\224!\377\340= \001\202\223\241", phdr = 0x1800034, phsize = 224, interp = 0x1800114 "/libexec/ld-elf.so.1", stack_flags = 6, tlsindex = 0, tlsinit = 0x0,
tlsinitsize = 0, tlssize = 0, tlsoffset = 0, tlsalign = 0, relro_page = 0x0, relro_size = 0, pltgot = 0x18186f4, rel = 0x0, relsize = 0, rela = 0x1801ebc, relasize = 1008, pltrel = 0x0,
pltrelsize = 0, pltrela = 0x1801f28, pltrelasize = 900, symtab = 0x1800798, strtab = 0x1801418 "", strsize = 2241, verneed = 0x1801e6c, verneednum = 1, verdef = 0x0, verdefnum = 0,
versyms = 0x1801cda, buckets = 0x1800164, nbuckets = 197, chains = 0x1800478, nchains = 200, nbuckets_gnu = 0, symndx_gnu = 0, maskwords_bm_gnu = 0, shift2_gnu = 0, dynsymcount = 200,
bloom_gnu = 0x0, buckets_gnu = 0x0, chain_zero_gnu = 0x0, rpath = 0x0, runpath = 0x0, needed = 0x41042010, needed_filtees = 0x0, needed_aux_filtees = 0x0, names = {stqh_first = 0x0,
stqh_last = 0x410410f8}, vertab = 0x41043080, vernum = 6, init = 25174700, fini = 25197908, preinit_array = 0, init_array = 0, fini_array = 0, preinit_array_num = 0, init_array_num = 0,
fini_array_num = 0, osrel = 1200084, mainprog = 1 '\001', rtld = 0 '\000', relocated = 1 '\001', ver_checked = 1 '\001', textrel = 0 '\000', symbolic = 0 '\000', bind_now = 0 '\000',
traced = 0 '\000', jmpslots_done = 0 '\000', init_done = 0 '\000', tls_done = 1 '\001', phdr_alloc = 0 '\000', z_origin = 0 '\000', z_nodelete = 0 '\000', z_noopen = 0 '\000',
z_loadfltr = 0 '\000', z_interpose = 0 '\000', z_nodeflib = 0 '\000', z_global = 0 '\000', ref_nodel = 0 '\000', init_scanned = 0 '\000', on_fini_list = 0 '\000', dag_inited = 0 '\000',
filtees_loaded = 0 '\000', irelative = 0 '\000', gnu_ifunc = 0 '\000', non_plt_gnu_ifunc = 0 '\000', crt_no_init = 1 '\001', valid_hash_sysv = 1 '\001', valid_hash_gnu = 0 '\000',
dlopened = 0 '\000', marker = 0 '\000', unholdfree = 0 '\000', doomed = 0 '\000', linkmap = {l_addr = 0x1800000 "\177ELF\001\002\001\t", l_name = 0x41042000 "/bin/ls", l_ld = 0x1818554,
l_next = 0x41041334, l_prev = 0x0}, dldags = {stqh_first = 0x0, stqh_last = 0x41041148}, dagmembers = {stqh_first = 0x0, stqh_last = 0x41041150}, dev = 0, ino = 0, priv = 0x0}
===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)
More information about the svn-src-head
mailing list