port's svn commit: r413746 - in head "many ports: mark broken on powerpc64": for what toolchains?

Sun Apr 24 00:21:05 UTC 2016

On 2016-Apr-23, at 4:17 PM, Steve Wills <swills at FreeBSD.org> wrote:
> 
> Hi,
> 
> On 04/23/16 05:50 PM, Mark Millard wrote:
>> Recently a large block of ports (including lang/gcc6-devel) were
>> marked in their Makefiles with
>> 
>>> BROKEN_powerpc64=       Does not build
>> 
>> 
>> Does this only apply for use of gcc/g++ 4.2.1 or specific other
>> verions? If so that is not the only way to have a powerpc64
>> environment. For example: how "official" is use of
>> devel/powerpc64-xtoolchain-gcc (so devel/powerpc64-gcc as well) and
>> lang/gcc49 used on a powerpc64 box (in my case an old PowerMac)? 
> 
> 
> The intent was just that they don't build with the default config, ie,
> gcc 4.2.1 from base. If there are ways to make things build with other
> compilers, we should look at getting those changes into ports.
> 
> You can find the logs of the build that I based all those changes on here:
> 
> http://poudriere.mouf.net/karl/poudriere/build.html?mastername=headpowerpc-default&build=2016-04-02_20h57m16s
> 
> Specifically the gcc6-devel one is here:
> 
> http://poudriere.mouf.net/karl/poudriere/data/headpowerpc-default/2016-04-02_20h57m16s/logs/errors/gcc6-devel-6.0.0.s20160320.log
> 
> Though I realize now that was simply a timeout and perhaps shouldn't
> have been marked as broken. A few of the llvm ones timed out and I
> didn't mark those as broken and meant to not mark any that timed out as
> broken, and to go back and retry them, but didn't notice this one was a
> timeout. We can remove the BROKEN_powerpc64 from lang/gcc-devel if it
> builds for you.

Since my original note -r413919 has updated devel/gcc6-devel. So I'll end up testing that update once I get to that point.

I'll note that the svn-ports-head entry for -r412746 lists several llvm items:

  head/devel/llvm-cheri/Makefile
  head/devel/llvm-devel/Makefile
  head/devel/llvm33/Makefile
  head/devel/llvm34/Makefile
  head/devel/llvm38/Makefile

But I do not build any of those normally. llvm38 likely is toolchain vintage dependent for if it builds or not. Others may be as well.

> Also note that build was using the 2016Q2 branch, but the
> BROKEN_powerpc64 was committed to the main branch. Perhaps that change
> should be merged to the 2016Q2 branch.
> 
> Anyway, I'm currently retrying the build, after fixing pixman and
> merging that to the 2016Q2 and then locally patching in the
> BROKEN_powerpc64 things into my local tree. You can see the progress of
> that here:
> 
> http://poudriere.mouf.net/karl/poudriere/build.html?mastername=headpowerpc-default&build=2016-04-21_17h38m55s
> 
> 
>> Can broken be marked for only specific tool chains?
>> 
> 
> Not trivially, but it's probably doable somehow.
> 
>>> # freebsd-version -ku 11.0-CURRENT 11.0-CURRENT # uname -aKU 
>>> FreeBSD FBSDG5C0 11.0-CURRENT FreeBSD 11.0-CURRENT #29 r297769M:
>>> Sat Apr  9 22:22:19 PDT 2016
>>> root at FBSDG5C0:/usr/obj/xtoolchain/powerpc.powerpc64/usr/src/sys/GENERIC64vtsc-NODEBUG
>>> powerpc 1100105 1100105
>> 
>> For the rest of this note I'll focus on lang/gcc-devel since I happen
>> to build and install it.
>> 
>>> # pkg info '*gcc*' gcc49-4.9.4.s20160406 
>>> gcc6-devel-6.0.0.s20160410 powerpc64-gcc-5.3.0 
>>> powerpc64-xtoolchain-gcc-0.1
>> 
>> gcc6-devel-6.0.0.s20160410 was built from -r413230 svn source on the
>> that same old PowerMac. -r413188 is the prior lang/gcc-devel check in
>> before being marked broken for powerpc64. For how I build it was not
>> broken at the time.
>> 
>> I historically use devel/powerpc64-xtoolchain-gcc (so
>> devel/powerpc64-gcc as well) for the so-called "cross build" stages
>> of buildworld/buildkernel (11.0-CURRENT). (I also do true cross
>> builds for TARGET_ARCH=powerpc64 from an amd64 context sometimes.)
>> 
>> gcc 4.2.1 is not present before, during, or after on the old
>> PowerMac. I do build clang and various related materials but do not
>> use clang for buildworld/buildkernel related activity. I experiment
>> with using clang for other things at times.
>> 
>> I historically use lang/gcc49 on the old PowerMac for:
>> 
>> A) building devel/powerpc64-gcc (not the only way to try to build
>> it) B) the "host" stages of buildworld/buildkernel (11.0-CURRENT) C)
>> building lang/gcc6-devel (not the only way to try to build it)
>> 
>> [I do not have lang/gcc5 built/installed only because it and
>> devel/powerpc64-gcc have file conflicts when they are based on the
>> same gcc version. Getting devel/powerpc64-gcc to build and install on
>> a powerpc64 requires some workarounds because it is not a true cross
>> compile environment and some file names are odd for self-hosted and
>> so not found during staging's copy activities.]
>> 
>> [I do build the system's clang 3.8.0 but do not use it other than for
>> exploring/checking its current powerpc64 FreeBSD limitations. It does
>> work for various things but not everything. Those experiments do not
>> include buildworld or buildkernel. For example: clang 3.8.0 targeting
>> powerpc64 can not build libstand due to lack of software floating
>> point support. C++ exception handling built via clang for powerpc64
>> is messed up as well.]
>> 
>> 
> 
> It would be nice if we could fix those things and get powerpc(64) built
> with clang.

https://llvm.org/bugs/show_bug.cgi?id=25780 [a meta-list of reports that block use by freebsd] lists various powerpc and powerpc64 code-generation issues for clang 3.8.0 vs. FreeBSD. As I remember each of the following includes examples of powerpc64 code-generation issues, not just powerpc (non-64) ones.

https://llvm.org/bugs/show_bug.cgi?id=26970
https://llvm.org/bugs/show_bug.cgi?id=26856
https://llvm.org/bugs/show_bug.cgi?id=26844
https://llvm.org/bugs/show_bug.cgi?id=26761

(I also submitted reports to freebsd as well so there would be reminders to track clang fixes if they ever occur. There may be more issues than I reported in either place.)

As I remember I thought there were also some FreeBSD problems that would be present after the above were fixed but the  code generation problems made it harder to be sure. Even if I was correct any testing based on the bad code-generation by clang in overlapping areas would not work. I'd prefer to recheck/reanalyze based on seeing good code generation results.

Unfortunately while I can slowly analyze/research the code generated it would take a lot longer to get the point of doing non-trival work on llvm or clang itself. And since somewhat after clang 3.8.0 moved into 11.0-CURRENT other things have kept me mostly out of clang investigations. I've just been rebuilding FreeBSD and some ports periodically in order to avoid large jumps later.

>> 
>> I have started a powerpc64 self-hosted buildworld/buildkernel for an
>> update to 11.0-CURRENT -r298518 in preparation for then updating my
>> ports to -r413907 or so.
>> 
>> My intent is to comment out the BROKEN_powerpc64 line in
>> lang/gcc6-devel/Makefile and see if lang/gcc6-devel still (re-)builds
>> once everything else is up to date.
>> 
>> But the builds involved take many hours so I'll likely not have a
>> result to report until tomorrow (or later).
>> 
>> 
>> Supporting example details:
>> 
>>> # svnlite info /usr/ports Path: /usr/ports Working Copy Root Path:
>>> /usr/ports URL: https://svn0.us-west.freebsd.org/ports/head 
>>> Relative URL: ^/head Repository Root:
>>> https://svn0.us-west.freebsd.org/ports Repository UUID:
>>> 35697150-7ecd-e111-bb59-0022644237b5 Revision: 413230 Node Kind:
>>> directory Schedule: normal Last Changed Author: kmoore Last Changed
>>> Rev: 413230 Last Changed Date: 2016-04-13 13:07:37 -0700 (Wed, 13
>>> Apr 2016)
>> 
>> (I'll svnlite update -r413907 /usr/ports [or there about] after the
>> system is updated. The above was used for my existing lang/gcc-devel
>> build on powerpc64 for powerpc64.)
>> 
>>> # more /etc/make.conf DEFAULT_VERSIONS+=perl5=5.22 
>>> WRKDIRPREFIX=/usr/obj/portswork WITH_DEBUG= WITH_DEBUG_FILES= 
>>> MALLOC_PRODUCTION= # # # For trying gcc49... # 
>>> CC=/usr/local/bin/gcc49 CXX=/usr/local/bin/g++49 
>>> CPP=/usr/local/bin/cpp49 # # # For trying port binutils... # 
>>> CROSS_BINUTILS_PREFIX=/usr/local/powerpc64-portbld-freebsd11.0/bin/
>>> 
>>> 
> AS=/usr/local/powerpc64-portbld-freebsd11.0/bin/as
>>> AR=/usr/local/powerpc64-portbld-freebsd11.0/bin/ar 
>>> LD=/usr/local/powerpc64-portbld-freebsd11.0/bin/ld 
>>> NM=/usr/local/powerpc64-portbld-freebsd11.0/bin/nm 
>>> OBJCOPY=/usr/local/powerpc64-portbld-freebsd11.0/bin/objcopy 
>>> OBJDUMP=/usr/local/powerpc64-portbld-freebsd11.0/bin/objdump 
>>> RANLIB=/usr/local/powerpc64-portbld-freebsd11.0/bin/ranlib 
>>> SIZE=/usr/local/powerpc64-portbld-freebsd11.0/bin/size #NO-SUCH:
>>> STRINGS=/usr/local/powerpc64-portbld-freebsd11.0/bin/strings 
>>> STRINGS=/usr/local/bin/strings
>> 
>> The above sort of make.conf is used for port builds, though I may
>> edit the details to control my experiments.
>> 
>> The below are tied to buildworld/buildkernel related activity:
>> 
>>> # more
>>> /root/sys_build_scripts.powerpc64-host/make_powerpc64vtsc_nodebug_incl_clang_xtoolchain-powerpc64-host.sh
>>> script
>>> /root/sys_typescripts/typescript_make_powerpc64vtsc_nodebug_incl_clang_xtoolchain-powerpc64-host-$(date
>>> +%Y-%m-%d:%H:%M:%S) \ env __MAKE_CONF="/root/src.configs/make.conf"
>>> SRC_ENV_CONF="/root/src.configs/src.conf.powerpc64-xtoolchain.powerpc64-host"
>>> \ MAKEOBJDIRPREFIX="/usr/obj/xtoolchain/powerpc.powerpc64" \ make
>>> $*
>> 
>> 
>> /root/src.configs/make.conf (used for buildworld/buildkernel) is
>> normally empty.
>> 
>>> # more
>>> /root/src.configs/src.conf.powerpc64-xtoolchain.powerpc64-host 
>>> TO_TYPE=powerpc64 TOOLS_TO_TYPE=${TO_TYPE} FROM_TYPE=powerpc64 
>>> TOOLS_FROM_TYPE=${FROM_TYPE} VERSION_CONTEXT=11.0 # 
>>> KERNCONF=GENERIC64vtsc-NODEBUG TARGET=powerpc .if ${.MAKE.LEVEL} ==
>>> 0 TARGET_ARCH=${TO_TYPE} .export TARGET_ARCH .endif # 
>>> WITHOUT_CROSS_COMPILER= # WITH_LIBCPLUSPLUS= WITH_BOOT= 
>>> #WITH_LIB32= WITH_CLANG= WITH_CLANG_IS_CC= WITH_CLANG_FULL= 
>>> WITH_LLDB= # # LIB32 builds but does not work via gcc variants
>>> [crtbeginS code problem(s)] WITHOUT_LIB32= WITHOUT_GCC= 
>>> WITHOUT_GNUCXX= # NO_WERROR= MALLOC_PRODUCTION= # 
>>> WITH_DEBUG_FILES= # # # For TO (so-called "cross") stages . . . #
>>> So-called-cross via ${TO_TYPE}-xtoolchain-gcc/${TO_TYPE}-gcc. . . #
>>> TOOLS_TO_TYPE based on ${TO_TYPE}-xtoolchain-gcc related bintutils.
>>> . . # CROSS_TOOLCHAIN=${TO_TYPE}-gcc X_COMPILER_TYPE=gcc 
>>> CROSS_BINUTILS_PREFIX=/usr/local/${TOOLS_TO_TYPE}-freebsd/bin/ .if
>>> ${.MAKE.LEVEL} == 0 
>>> XCC=/usr/local/bin/${TOOLS_TO_TYPE}-portbld-freebsd${VERSION_CONTEXT}-gcc
>>> 
>>> 
> XCXX=/usr/local/bin/${TOOLS_TO_TYPE}-portbld-freebsd${VERSION_CONTEXT}-g++
>>> XCPP=/usr/local/bin/${TOOLS_TO_TYPE}-portbld-freebsd${VERSION_CONTEXT}-cpp
>>> 
>>> 
> .export XCC
>>> .export XCXX .export XCPP 
>>> XAS=/usr/local/${TOOLS_TO_TYPE}-freebsd/bin/as 
>>> XAR=/usr/local/${TOOLS_TO_TYPE}-freebsd/bin/ar 
>>> XLD=/usr/local/${TOOLS_TO_TYPE}-freebsd/bin/ld 
>>> XNM=/usr/local/${TOOLS_TO_TYPE}-freebsd/bin/nm 
>>> XOBJCOPY=/usr/local/${TOOLS_TO_TYPE}-freebsd/bin/objcopy 
>>> XOBJDUMP=/usr/local/${TOOLS_TO_TYPE}-freebsd/bin/objdump 
>>> XRANLIB=/usr/local/${TOOLS_TO_TYPE}-freebsd/bin/ranlib 
>>> XSIZE=/usr/local/${TOOLS_TO_TYPE}-freebsd/bin/size #NO-SUCH:
>>> XSTRINGS=/usr/local/${TOOLS_TO_TYPE}-freebsd/bin/strings 
>>> XSTRINGS=/usr/local/bin/${TOOLS_TO_TYPE}-freebsd-strings .export
>>> XAS .export XAR .export XLD .export XNM .export XOBJCOPY .export
>>> XOBJDUMP .export XRANLIB .export XSIZE .export XSTRINGS .endif # # 
>>> # For FROM (host) stages . . . # From gccXY (such as gcc49 but not
>>> xtoolchain) # TOOLS_FROM_TYPE's appropriate binutils. . . # .if
>>> ${.MAKE.LEVEL} == 0 CC=env C_INCLUDE_PATH=/usr/include
>>> /usr/local/bin/gcc49 -L/usr/lib CXX=env C_INCLUDE_PATH=/usr/include
>>> CPLUS_INCLUDE_PATH=/usr/include/c++/v1 /usr/local/bin/g++49
>>> -std=c++11 -nostdinc++ -L/usr/lib CPP=/usr/local/bin/cpp49 .export
>>> CC .export CXX .export CPP 
>>> AS=/usr/local/${TOOLS_FROM_TYPE}-portbld-freebsd${VERSION_CONTEXT}/bin/as
>>> 
>>> 
> AR=/usr/local/${TOOLS_FROM_TYPE}-portbld-freebsd${VERSION_CONTEXT}/bin/ar
>>> LD=/usr/local/${TOOLS_FROM_TYPE}-portbld-freebsd${VERSION_CONTEXT}/bin/ld
>>> 
>>> 
> NM=/usr/local/${TOOLS_FROM_TYPE}-portbld-freebsd${VERSION_CONTEXT}/bin/nm
>>> OBJCOPY=/usr/local/${TOOLS_FROM_TYPE}-portbld-freebsd${VERSION_CONTEXT}/bin/objcopy
>>> 
>>> 
> OBJDUMP=/usr/local/${TOOLS_FROM_TYPE}-portbld-freebsd${VERSION_CONTEXT}/bin/objdump
>>> RANLIB=/usr/local/${TOOLS_FROM_TYPE}-portbld-freebsd${VERSION_CONTEXT}/bin/ranlib
>>> 
>>> 
> SIZE=/usr/local/${TOOLS_FROM_TYPE}-portbld-freebsd${VERSION_CONTEXT}/bin/size
>>> #NO-SUCH:
>>> STRINGS=/usr/local/${TOOLS_FROM_TYPE}-portbld-freebsd${VERSION_CONTEXT}/bin/strings
>>> 
>>> 
> STRINGS=/usr/local/bin/strings
>>> .export AS .export AR .export LD .export NM .export OBJCOPY .export
>>> OBJDUMP .export RANLIB .export SIZE .export STRINGS .endif
>> 
>>> # svnlite status /usr/src ?       /usr/src/.snap M
>>> /usr/src/contrib/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
>>> 
>>> 
> M       /usr/src/lib/csu/powerpc64/Makefile
>>> ?       /usr/src/restoresymtable ?
>>> /usr/src/sys/arm/conf/RPI2-NODBG M
>>> /usr/src/sys/boot/ofw/Makefile.inc M
>>> /usr/src/sys/boot/powerpc/Makefile M
>>> /usr/src/sys/boot/powerpc/Makefile.inc M
>>> /usr/src/sys/boot/uboot/Makefile.inc M
>>> /usr/src/sys/conf/Makefile.powerpc M
>>> /usr/src/sys/conf/kern.mk M       /usr/src/sys/conf/kmod.mk ?
>>> /usr/src/sys/powerpc/conf/GENERIC64-NODBG ?
>>> /usr/src/sys/powerpc/conf/GENERIC64vtsc ?
>>> /usr/src/sys/powerpc/conf/GENERIC64vtsc-NODEBUG ?
>>> /usr/src/sys/powerpc/conf/GENERICvtsc ?
>>> /usr/src/sys/powerpc/conf/GENERICvtsc-NODEBUG M
>>> /usr/src/sys/powerpc/ofw/ofw_machdep.c M
>>> /usr/src/sys/powerpc/powerpc/exec_machdep.c
>> 
>> 
> 
> Let me know what you find. There was some work on external compiler
> support, though I don't know it's current status.

My powerpc64 FreeBSD activity is completely dependent on the external compiler support: I'm using FreeBSD's modern libc++, not libstdc++. But there are some workarounds involved for my complete avoidance of gcc 4.2.1. I've a few other work arounds for the PowerMac context as well. For example: downloaded release and snapshot installations do not reliably boot such PowerMacs but instead frequently hang during boot. (The frequency may depend on the amount of RAM.)

I'll let you know how the updated devel/gcc6-devel goes for my style of powerpc64 environment.

If a devel/gcc6 appears I'll likely build it at some point.

> Steve

===
Mark Millard
markmi at dsl-only.net