FYI: Things that I've noticed for powerpc64-xtoolchain-gcc...

[Mark Millard]

You [Craig Rodrigues] seem to be checking some ??-xtoolchain-gcc things officially. I've some user-experience notes from happening to use powerpc64-xtoolchain-gcc in my exploration of FreeBSD. Hopefully some of it might be of interest. It groups together information from various notices as I was doing the explorations.

I've been exploring FreeBSD, in part using powerpc64-xtoolchain-gcc --but on a powerpc64 11.0-CURRENT environment for the points at hand: A form of self-hosting xtoolchain use. As stands I'm limited to powerpc64 and powerpc contexts for FreeBSD, PowerMacs specifically. So I might over infer general ??-xtoolchain-gcc properties.

My use is not generally of the grab-source and buildworld/buildkernel from scratch kind of context. The closest I came to that was rebuilding a gcc 4.2.1 built powerpc64 11.0-CURRENT to be a powerpc64-gcc based 11.0-current that had libc++ in use.

After that I've updated (svnlite update -r??????) and rebuilt a newer 11.0-CURRENT.

I'm not sure if the continuous integration or other ??-xtoolchain-gcc testing checks on rebuild-for-update kind of context or not. One of the things of note for me is specific to that kind of context where headers and libraries temporarily have both old and new vintages around and the right ones need to be used in specific stages. Other aspects of my use might also go outside the range of your activity as well.


The primary things that I've run into include:

A) Rebuilds from updated sources were picking up old headers (from, for example, /usr/include) and old libraries, not just for legacy/ where such makes sense.

B) I had a specific problem originally with atf-c++ finding the C++ standard library headers. I added a specific -I for what was ${WORLDTMP}/usr/include/c++/v1 in order for it to find the files. I also put in a matching -L for /usr/obj/usr/src/lib/libc++ .

C) powerpc64-gcc automatically looks in /usr/local/include for header files (like most such ports) and this can pick up the wrong file(s). I ended up using the manual policy of renaming /usr/local/include/iconv.h during buildworld/buildkernel so that it would not be found and used where the build should have found a file with different content from my /usr/src/... area (or its WORLDTMP copy).

D) I've been limited to WITHOUT_BOOT= WITHOUT_LIB32= by being unable to get the 32-bit environment activity to work: CROSS_TOOLCHAIN=powerpc64-gcc use did not allow the -melf32ppc_fbsd . Does targeting i386 for BOOT and LIB32 from amd64 have a similar problem with -melf_i386_fbsd ?

E) I've been limited to WITHOUT_GCC_BOOTSTRAP= by XCC being a full path making the build environment ignore WITH_GCC_BOOTSTRAP= . Similarly WITH_GCC= does not actually cause gcc 4.2.1 to be built. (I wanted WITHOUT_GNUCXX= in order to try libc++ only. So I've not tried WITH_GNUCXX= .)

F) I've been limited to WITHOUT_CLANG_BOOTSTRAP= WITHOUT_CLANG= WITHOUT_LLDB= by various issues when clang related builds are involved. I'll not go into details.

G) Last I tried it powerpc64-xtoolchain-gcc does not correctly install itself automatically when one attempts to install it in a powerpc64-host context. (It installs fine on a powerpc (non-64) host context.) This may be associated with the same points that make the distinction between /usr/obj/powerpc.powerpc64/usr/src/ (on/in powerpc (non-64)) and /usr/obj/usr/src/ (on/in powerpc64) as a default path: in part the errors were empty file name prefixes vs. filled-in prefixes.

H) After the gcc 4.2.1 -> 4.9.1 bootstrap I had to rebuild powerpc64-gcc in order to get the -pg compiles to always work: without that at least one required example of such a compile crashed the compiler every time it was tried. All -pg compiles worked fine after the rebuild. (I used gcc5 to rebuild 4.9.1 because of (G) ending up involved otherwise.) Before, when still booted from the gcc-4.2.1-based world, the original gcc 4.9.1 compiler did not crash for -pg compiles.

I) Trying to use powerpc64-xtoolchain-gcc from a powerpc (non-64) host does not work. One aspect is tool chain related: C/C++/assembler usage need to be told to be 64-bit such that, for example, __powerpc64__ ends up defined for pre-processor use. That is not happening and incorrect processor generated files result. For example the assembler reports syntax errors from lack of definitions related to TOC use for powerpc64.


There are some work arounds involved for things very powerpc64/powerpc context specific, such as some Makefile*'s forcing use of CC:=gcc . I've not noted such things above: The above targets what I'd guess might be more general issues. I've also not noted when 11.0-CURRENT has required a few files to have specific versions picked out that did not match the snapshot I was generally using at the time.


My current context for this is:

> # freebsd-version -ku; uname -apKU
> 11.0-CURRENT
> 11.0-CURRENT
> FreeBSD FBSDG5C0 11.0-CURRENT FreeBSD 11.0-CURRENT #0 r280598M: Fri Mar 27 18:26:08 PDT 2015     root at FBSDG5C0:/usr/obj/usr/src/sys/GENERIC64vtsc-NODEBUG  powerpc powerpc64 1100066 1100066


> # dmesg | head
> ...
> FreeBSD 11.0-CURRENT #0 r280598M: Fri Mar 27 18:26:08 PDT 2015
>     root at FBSDG5C0:/usr/obj/usr/src/sys/GENERIC64vtsc-NODEBUG powerpc
> gcc version 4.9.1 (FreeBSD Ports Collection for powerpc64) 
> ...

I originally bootstrapped using -r279514. After that I attempted rebuilding 279514 various times exploring what I could enable successfully vs. what I could not. -r280598 was my first attempt to svnlite upgrade then rebuild.

The 4.2.1 gcc is not present. Nor has clang been built: 3.4.1 was removed by make delete-old after the initial bootstrap. I'm using powerpc64-gcc as the only system compiler after the bootstrap. (This fits with (E) above.)

> make -j 8 CROSS_TOOLCHAIN=powerpc64-gcc \
> WITHOUT_CLANG_BOOTSTRAP= WITHOUT_CLANG= WITHOUT_CLANG_IS_CC= \
> WITHOUT_LLDB= \
> WITHOUT_GCC_BOOTSTRAP= WITHOUT_GCC= WITHOUT_GNUCXX= \
> WITHOUT_BOOT= WITHOUT_LIB32= \
> buildworld buildkernel \
> KERNCONF=GENERIC64vtsc-NODEBUG \
> TARGET=powerpc TARGET_ARCH=powerpc64

For the below: Remember, no 4.2.1 gcc and no clang. So powerpc64-gcc also has to cover what those would normally be used for, not just the XCC, XCXX, XCPP usage. (powerpc and powerpc64 via clang is not a supported combination yet anyway.)

> # more /etc/src.conf 
> NO_WERROR=
> WITH_LIBCPLUSPLUS=
> CC=/usr/local/bin/powerpc64-portbld-freebsd11.0-gcc
> CXX=/usr/local/bin/powerpc64-portbld-freebsd11.0-g++
> CPP=/usr/local/bin/powerpc64-portbld-freebsd11.0-cpp
> CROSS_BINUTILS_PREFIX=/usr/local/powerpc64-freebsd/bin/
> X_COMPILER_TYPE=gcc
> #CFLAGS+=-L/usr/obj/usr/src/tmp/usr/lib/.
> # The order of the two CXXFLAGS additions below is important.
> CXXFLAGS+=-I/usr/obj/usr/src/tmp/usr/include/c++/v1/. -std=gnu++11 -L/usr/obj/usr/src/lib/libc++/.
> CXXFLAGS+=-I/usr/include/c++/v1/. -std=gnu++11 -L/usr/lib/.

(I make no claim that the above is a general solution but with some care about how to do builds it has allowed the experiments that I've done. I sometimes have used /usr/srcC/ instead of /usr/src/ : I have two 11.0-CURRENT source trees.)

> # more /etc/make.conf 
> WRKDIRPREFIX=/usr/obj/portswork
> #WITH_DEBUG=
> MALLOC_PRODUCTION=

> # svnlite st /usr/src --no-ignore
> ?       /usr/src/.snap
> ?       /usr/src/restoresymtable
> M       /usr/src/sys/ddb/db_main.c
> M       /usr/src/sys/ddb/db_script.c
> ?       /usr/src/sys/powerpc/conf/GENERIC64vtsc
> ?       /usr/src/sys/powerpc/conf/GENERIC64vtsc-NODEBUG
> ?       /usr/src/sys/powerpc/conf/GENERICvtsc
> ?       /usr/src/sys/powerpc/conf/GENERICvtsc-NODEBUG
> M       /usr/src/sys/powerpc/ofw/ofw_machdep.c
> M       /usr/src/sys/powerpc/ofw/ofwcall64.S

All of the above files that are relevant existed and were in use long before I started the powerpc64-xtoolchain-gcc related explorations. ofw_machdep.c is tied to a PowerMac-specific change for reliable booting. The db_'s and ofwcall64.S are tied to getting information from early-boot crashes if I get any more. The GENERIC*'s relate to that early-information gathering and to disabling ps3 so that I can then also have both vt and sc enabled.




As this has been an exploration of the unfamiliar, I've had false starts, backtracking, and the like. While I've learned a bunch I doubt that I could start over and get to where I am by a step-by-step procedure that avoided backtracking, pondering, retries, and so on. In some respects I've been lucky that certain types of changes did not happen between my original and upgraded vintages: things would not have worked that did work.

===
Mark Millard
markmi at dsl-only.net