Bug 205663 Clang getting Bus Errors (arm SCLTR Bit[12]==1 context): Reported fixed on llvm's trunk

Sun Jan 10 17:59:37 UTC 2016

[A top post of my so far successful experiment. . .]

amd64 -> rpi2 cross build: buildworld buildkernel installkernel installworld mergemaster delete-old then boot the result on an rpi2 worked ( -march=armv7a -mcpu=cortex-a7 -mfloat-abi=softfp -mno-unaligned-access used ).

In order to also build lldb I used the following beyond what Ian sent out for -mlong-calls usage:
(I make no claim to have tested the lldb build.)
(Tabs likely not preserved in the copy/paste operations.)

> # more lib_libc++_Makfile.diff 
> Index: /usr/src/lib/libc++/Makefile
> ===================================================================
> --- /usr/src/lib/libc++/Makefile        (revision 293430)
> +++ /usr/src/lib/libc++/Makefile        (working copy)
> @@ -57,6 +57,9 @@
>  .endfor
>  
>  WARNS=         0
> +.if ${MACHINE_CPUARCH} == "arm"
> +STATIC_CXXFLAGS+=-mlong-calls
> +.endif
>  CFLAGS+=       -I${HDRDIR} -I${_LIBCXXRTDIR} -nostdlib -DLIBCXXRT
>  .if empty(CXXFLAGS:M-std=*)
>  CXXFLAGS+=     -std=c++11

> # more usr_bin_clang_lldb_Makefile.diff 
> Index: /usr/src/usr.bin/clang/lldb/Makefile
> ===================================================================
> --- /usr/src/usr.bin/clang/lldb/Makefile        (revision 293430)
> +++ /usr/src/usr.bin/clang/lldb/Makefile        (working copy)
> @@ -6,6 +6,10 @@
>  
>  LLDB_SRCS=${.CURDIR}/../../../contrib/llvm/tools/lldb
>  
> +.if ${MACHINE_CPUARCH} == "arm"
> +CFLAGS+=-mlong-calls
> +.endif
> +
>  CFLAGS+=       -I${LLDB_SRCS}/include
>  CXXFLAGS+=     -std=c++11

The rpi2 is now attempting buildworld buildkernel targeting itself. It is way beyond where clang 3.7.1 crashed.

The above makes no claim at investigating or addressing powerpc64 (PowerMac G5) or powerpc (PowerMac G4/G3) yet(?). So far as I know clang mixed with the available supporting toolchain alternatives are still insufficient for targeting those for a WITH_LIBCPLUSPLUS= build. Instead something like powerpc64-xtoolchain-gcc/powerpc64-gcc must be used, even if clang is built. (Last I knew gcc5 variations/vintages would reject some of the WITH_CLANG_EXTRAS= code --but I've not tried based on clang 3.8.0 materials.)

===
Mark Millard
markmi at dsl-only.net

On 2016-Jan-10, at 3:55 AM, Mark Millard <markmi at dsl-only.net> wrote:
> 
> 
> On 2016-Jan-9, at 10:55 AM, Ian Lepore <ian at freebsd.org> wrote:
>> 
>> On Sat, 2016-01-09 at 15:03 +0100, Dimitry Andric wrote:
>>> On 09 Jan 2016, at 04:46, Mark Millard <markmi at dsl-only.net> wrote:
>>>> 
>>>> On 2016-Jan-7, at 2:57 PM, Dimitry Andric <dim at FreeBSD.org>
>>>> wrote:
>>> ...
>>>>> FYI, I have added a -mno-movt option for this purpose upstream,
>>>>> and
>>>>> imported a newer snapshot into the clang380-import branch.  As of
>>>>> r293384, it now uses the new option spelling for modules, if your
>>>>> clang
>>>>> is 3.8.0 or higher.
>>>>> 
>>>>> -Dimitry
>>>> 
>>>> I've not been able to get to the point of running clang++ 3.8 on
>>>> the rpi2 yet: R_ARM_CALL and R_ARM_JUMP24 relocation truncations
>>>> during the cross build's buildworld interfere.
>>> 
>>> Yes, this is caused by too large call distances.  In other words, the
>>> clang executable is getting to big to link.  Apparently we need to do
>>> some tricks with -mlongcall to fix this.  As I am no arm expert, I
>>> welcome any patch submissions. :-)
>>> 
>>> -Dimitry
>>> 
>> 
>> Here's the patch I got from Andy for the clang380 branch, modified with
>> Warner's suggestion to use MACHINE_CPUARCH instead of MACHINE.  With
>> this I can get a working arm world that will build a runnable
>> helloworld.c (and .cc) on a dreamplug.  (I.e., it appears clang 3.8.0
>> fixes the problem we had with clang 3.7.x where it wouldn't run at all
>> on armv4/5 systems).  I have not tried compling anything complex yet.
>> 
>> -- Ian
> 
> 
> Context: When I build I normally build lldb and the like as well, even using WITH_CLANG_EXTRAS= .
> 
> In trying to get lldb to link I eventually get to the point that libc++ is getting relocation truncations. Before getting to that I deal with /usr/src/usr.bin/clang/lldb/Makefile to cover what is initially reported during buildworld for lldb relocation truncations. Then with that in place and retrying I get reports from libc++.a for a couple of the contained .o files having relocations that are truncated (before it reports "additional relocation overflows omitted from the output"):
> 
> /usr/obj/clang/arm.armv6/usr/src/tmp/usr/lib/libc++.a(thread.o)
> /usr/obj/clang/arm.armv6/usr/src/tmp/usr/lib/libc++.a(locale.o)
> 
> (Both have messages from multiple places in each .o file.)
> 
> As far as I can tell for general use -long-calls is going to be needed for at least some system level .a library content if the .a's are to be used.
> 
> Which leaves me wondering if STATIC_CXXFLAGS having -mlong-calls for arm system libraries fairly generally is appropriate for those intending on building the arm-native clang toolchain  and related material in buildworld. (STATIC_CFLAGS too?) A significant case analysis of what happens to currently be too far apart would be fragile as things grow even more later.
> 
> This sort of issue may well not be limited to TARGET=arm contexts.
> 
> The detailed libc++.a relocation truncation complaints that I got were:
> 
> 
>> --- all_subdir_lldb ---
>> /usr/obj/clang/arm.armv6/usr/src/tmp/usr/lib/libc++.a(thread.o): In function `_ZNSt3__16vectorINS_4pairIPNS_18condition_variableEPNS_5mutexEEENS_18__hidden_allocatorIS6_EEE21__push_back_slow_pathIS6_EEvOT_':
>> /usr/src/lib/libc++/../../contrib/libc++/src/thread.cpp:(.text._ZNSt3__16vectorINS_4pairIPNS_18condition_variableEPNS_5mutexEEENS_18__hidden_allocatorIS6_EEE21__push_back_slow_pathIS6_EEvOT_[_ZNSt3__16vectorINS_4pairIPNS_18condition_variableEPNS_5mutexEEENS_18__hidden_allocatorIS6_EEE21__push_back_slow_pathIS6_EEvOT_]+0x30): relocation truncated to fit: R_ARM_CALL against symbol `std::__1::__vector_base_common<true>::__throw_length_error() const' defined in .text._ZNKSt3__120__vector_base_commonILb1EE20__throw_length_errorEv[_ZNKSt3__120__vector_base_commonILb1EE20__throw_length_errorEv] section in /usr/obj/clang/arm.armv6/usr/src/usr.bin/clang/lldb/../../../lib/clang/liblldbCore/liblldbCore.a(CxaDemangle.o)
> 
>> /usr/obj/clang/arm.armv6/usr/src/tmp/usr/lib/libc++.a(thread.o): In function `_ZNSt3__16vectorIPNS_17__assoc_sub_stateENS_18__hidden_allocatorIS2_EEE21__push_back_slow_pathIRKS2_EEvOT_':
>> /usr/src/lib/libc++/../../contrib/libc++/src/thread.cpp:(.text._ZNSt3__16vectorIPNS_17__assoc_sub_stateENS_18__hidden_allocatorIS2_EEE21__push_back_slow_pathIRKS2_EEvOT_[_ZNSt3__16vectorIPNS_17__assoc_sub_stateENS_18__hidden_allocatorIS2_EEE21__push_back_slow_pathIRKS2_EEvOT_]+0x30): relocation truncated to fit: R_ARM_CALL against symbol `std::__1::__vector_base_common<true>::__throw_length_error() const' defined in .text._ZNKSt3__120__vector_base_commonILb1EE20__throw_length_errorEv[_ZNKSt3__120__vector_base_commonILb1EE20__throw_length_errorEv] section in /usr/obj/clang/arm.armv6/usr/src/usr.bin/clang/lldb/../../../lib/clang/liblldbCore/liblldbCore.a(CxaDemangle.o)
> 
> . . .
>> --- all_subdir_clang ---
>> /usr/obj/clang/arm.armv6/usr/src/tmp/usr/lib/libc++.a(locale.o): In function `std::__1::collate_byname<char>::do_compare(char const*, char const*, char const*, char const*) const':
> . . .
>> --- all_subdir_clang ---
>> /usr/src/lib/libc++/../../contrib/libc++/src/locale.cpp:(.text+0x81ec): relocation truncated to fit: R_ARM_CALL against symbol `std::__1::__basic_string_common<true>::__throw_length_error() const' defined in .text._ZNKSt3__121__basic_string_commonILb1EE20__throw_length_errorEv[_ZNKSt3__121__basic_string_commonILb1EE20__throw_length_errorEv] section in /usr/obj/clang/arm.armv6/usr/src/usr.bin/clang/lldb/../../../lib/clang/liblldbCore/liblldbCore.a(CxaDemangle.o)
> . . .
>> --- all_subdir_clang ---
>> /usr/src/lib/libc++/../../contrib/libc++/src/locale.cpp:(.text+0x82c0): relocation truncated to fit: R_ARM_CALL against symbol `std::__1::__basic_string_common<true>::__throw_length_error() const' defined in .text._ZNKSt3__121__basic_string_commonILb1EE20__throw_length_errorEv[_ZNKSt3__121__basic_string_commonILb1EE20__throw_length_errorEv] section in /usr/obj/clang/arm.armv6/usr/src/usr.bin/clang/lldb/../../../lib/clang/liblldbCore/liblldbCore.a(CxaDemangle.o)
> 
>> /usr/obj/clang/arm.armv6/usr/src/tmp/usr/lib/libc++.a(locale.o): In function `std::__1::collate_byname<char>::do_transform(char const*, char const*) const':
>> /usr/src/lib/libc++/../../contrib/libc++/src/locale.cpp:(.text+0x8430): relocation truncated to fit: R_ARM_CALL against symbol `std::__1::__basic_string_common<true>::__throw_length_error() const' defined in .text._ZNKSt3__121__basic_string_commonILb1EE20__throw_length_errorEv[_ZNKSt3__121__basic_string_commonILb1EE20__throw_length_errorEv] section in /usr/obj/clang/arm.armv6/usr/src/usr.bin/clang/lldb/../../../lib/clang/liblldbCore/liblldbCore.a(CxaDemangle.o)
> 
>> /usr/obj/clang/arm.armv6/usr/src/tmp/usr/lib/libc++.a(locale.o): In function `std::__1::collate_byname<wchar_t>::do_compare(wchar_t const*, wchar_t const*, wchar_t const*, wchar_t const*) const':
> 
>> --- all_subdir_clang ---
>> /usr/src/lib/libc++/../../contrib/libc++/src/locale.cpp:(.text+0x88a8): relocation truncated to fit: R_ARM_CALL against symbol `std::__1::__basic_string_common<true>::__throw_length_error() const' defined in .text._ZNKSt3__121__basic_string_commonILb1EE20__throw_length_errorEv[_ZNKSt3__121__basic_string_commonILb1EE20__throw_length_errorEv] section in /usr/obj/clang/arm.armv6/usr/src/usr.bin/clang/lldb/../../../lib/clang/liblldbCore/liblldbCore.a(CxaDemangle.o)
> 
>> /usr/src/lib/libc++/../../contrib/libc++/src/locale.cpp:(.text+0x899c): relocation truncated to fit: R_ARM_CALL against symbol `std::__1::__basic_string_common<true>::__throw_length_error() const' defined in .text._ZNKSt3__121__basic_string_commonILb1EE20__throw_length_errorEv[_ZNKSt3__121__basic_string_commonILb1EE20__throw_length_errorEv] section in /usr/obj/clang/arm.armv6/usr/src/usr.bin/clang/lldb/../../../lib/clang/liblldbCore/liblldbCore.a(CxaDemangle.o)
> 
>> /usr/obj/clang/arm.armv6/usr/src/tmp/usr/lib/libc++.a(locale.o): In function `std::__1::collate_byname<wchar_t>::do_transform(wchar_t const*, wchar_t const*) const':
>> /usr/src/lib/libc++/../../contrib/libc++/src/locale.cpp:(.text+0x8b34): relocation truncated to fit: R_ARM_CALL against symbol `std::__1::__basic_string_common<true>::__throw_length_error() const' defined in .text._ZNKSt3__121__basic_string_commonILb1EE20__throw_length_errorEv[_ZNKSt3__121__basic_string_commonILb1EE20__throw_length_errorEv] section in /usr/obj/clang/arm.armv6/usr/src/usr.bin/clang/lldb/../../../lib/clang/liblldbCore/liblldbCore.a(CxaDemangle.o)
> 
>> /usr/obj/clang/arm.armv6/usr/src/tmp/usr/lib/libc++.a(locale.o): In function `std::__1::collate<char>::do_transform(char const*, char const*) const':
>> /usr/src/lib/libc++/../../contrib/libc++/src/locale.cpp:(.text._ZNKSt3__17collateIcE12do_transformEPKcS3_[_ZNKSt3__17collateIcE12do_transformEPKcS3_]+0x34): relocation truncated to fit: R_ARM_CALL against symbol `std::__1::__basic_string_common<true>::__throw_length_error() const' defined in .text._ZNKSt3__121__basic_string_commonILb1EE20__throw_length_errorEv[_ZNKSt3__121__basic_string_commonILb1EE20__throw_length_errorEv] section in /usr/obj/clang/arm.armv6/usr/src/usr.bin/clang/lldb/../../../lib/clang/liblldbCore/liblldbCore.a(CxaDemangle.o)
> 
>> /usr/obj/clang/arm.armv6/usr/src/tmp/usr/lib/libc++.a(locale.o): In function `std::__1::collate<wchar_t>::do_transform(wchar_t const*, wchar_t const*) const':
>> /usr/src/lib/libc++/../../contrib/libc++/src/locale.cpp:(.text._ZNKSt3__17collateIwE12do_transformEPKwS3_[_ZNKSt3__17collateIwE12do_transformEPKwS3_]+0x38): relocation truncated to fit: R_ARM_CALL against symbol `std::__1::__basic_string_common<true>::__throw_length_error() const' defined in .text._ZNKSt3__121__basic_string_commonILb1EE20__throw_length_errorEv[_ZNKSt3__121__basic_string_commonILb1EE20__throw_length_errorEv] section in /usr/obj/clang/arm.armv6/usr/src/usr.bin/clang/lldb/../../../lib/clang/liblldbCore/liblldbCore.a(CxaDemangle.o)
> 
>> /usr/obj/clang/arm.armv6/usr/src/tmp/usr/lib/libc++.a(locale.o): In function `std::__1::num_put<char, std::__1::ostreambuf_iterator<char, std::__1::char_traits<char> > >::do_put(std::__1::ostreambuf_iterator<char, std::__1::char_traits<char> >, std::__1::ios_base&, char, long) const':
>> /usr/src/lib/libc++/../../contrib/libc++/src/locale.cpp:(.text._ZNKSt3__17num_putIcNS_19ostreambuf_iteratorIcNS_11char_traitsIcEEEEE6do_putES4_RNS_8ios_baseEcl[_ZNKSt3__17num_putIcNS_19ostreambuf_iteratorIcNS_11char_traitsIcEEEEE6do_putES4_RNS_8ios_baseEcl]+0x16c): additional relocation overflows omitted from the output
> 
> 
> Ian's/Andy's original patch is listed below for reference.
> 
>> Index: lib/clang/clang.lib.mk
>> ===================================================================
>> --- lib/clang/clang.lib.mk	(revision 293584)
>> +++ lib/clang/clang.lib.mk	(working copy)
>> @@ -6,4 +6,8 @@ LLVM_SRCS= ${.CURDIR}/../../../contrib/llvm
>> 
>> INTERNALLIB=
>> 
>> +.if ${MACHINE_CPUARCH} == "arm"
>> +STATIC_CXXFLAGS+=-mlong-calls
>> +.endif
>> +
>> .include <bsd.lib.mk>
>> Index: lib/csu/arm/Makefile
>> ===================================================================
>> --- lib/csu/arm/Makefile	(revision 293584)
>> +++ lib/csu/arm/Makefile	(working copy)
>> @@ -23,7 +23,7 @@ CLEANFILES+=	crt1.s gcrt1.s Scrt1.s
>> # directly compiled to .o files.
>> 
>> crt1.s: crt1.c
>> -	${CC} ${CFLAGS} -S -o ${.TARGET} ${.CURDIR}/crt1.c
>> +	${CC} ${CFLAGS} -mlong-calls -S -o ${.TARGET} ${.CURDIR}/crt1.c
>> 	sed ${SED_FIX_NOTE} ${.TARGET}
>> 
>> crt1.o: crt1.s
>> @@ -30,7 +30,7 @@ crt1.o: crt1.s
>> 	${CC} ${ACFLAGS} -c -o ${.TARGET} crt1.s
>> 
>> gcrt1.s: crt1.c
>> -	${CC} ${CFLAGS} -DGCRT -S -o ${.TARGET} ${.CURDIR}/crt1.c
>> +	${CC} ${CFLAGS} -mlong-calls -DGCRT -S -o ${.TARGET} ${.CURDIR}/crt1.c
>> 	sed ${SED_FIX_NOTE} ${.TARGET}
>> 
>> gcrt1.o: gcrt1.s
>> Index: usr.bin/clang/clang/Makefile
>> ===================================================================
>> --- usr.bin/clang/clang/Makefile	(revision 293584)
>> +++ usr.bin/clang/clang/Makefile	(working copy)
>> @@ -11,7 +11,11 @@ SRCS=	cc1_main.cpp \
>> 
>> .if ${MK_SHARED_TOOLCHAIN} == "no"
>> NO_SHARED?= yes
>> +
>> +.if ${MACHINE_CPUARCH} == "arm"
>> +CFLAGS+=-mlong-calls
>> .endif
>> +.endif
>> 
>> LINKS=	${BINDIR}/clang ${BINDIR}/clang++ \
>> 	${BINDIR}/clang ${BINDIR}/clang-cpp
>> 
> 