[Bug 258494] lang/erlang lang/erlang-runtime21 lang/erlang-runtime23: clang 13 build breaks dtrace if PGO is enabled

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=258494

            Bug ID: 258494
           Summary: lang/erlang lang/erlang-runtime21
                    lang/erlang-runtime23: clang 13 build breaks dtrace if
                    PGO is enabled
           Product: Ports & Packages
           Version: Latest
          Hardware: Any
                OS: Any
            Status: New
          Severity: Affects Some People
          Priority: ---
         Component: Individual Port(s)
          Assignee: erlang@FreeBSD.org
          Reporter: dim@FreeBSD.org
          Assignee: erlang@FreeBSD.org
             Flags: maintainer-feedback?(erlang@FreeBSD.org)

During an exp-run for llvm 13 (see bug 258209), it turned out that lang/erlang
and lang/erlang-runtime2[1-3] fail to build with clang 13:

http://gohan04.nyi.freebsd.org/data/mainamd64PR258209-default/2021-09-05_20h27m09s/logs/errors/erlang-21.3.8.24_1,4.log
http://gohan04.nyi.freebsd.org/data/mainamd64PR258209-default/2021-09-05_20h27m09s/logs/errors/erlang-runtime21-21.3.8.24_1.log
http://gohan04.nyi.freebsd.org/data/mainamd64PR258209-default/2021-09-05_20h27m09s/logs/errors/erlang-runtime23-23.3.4.6.log

What appears to happen is that for these versions of erlang, PGO is enabled,
and it first builds a PGO-enabled beam.smp:

gmake[5]: Entering directory
'/wrkdirs/usr/ports/lang/erlang/work/otp-OTP-21.3.8.24/erts/emulator'
if utils/gen_git_version amd64-portbld-freebsd14.0/gen_git_version.mk; then
touch beam/erl_bif_info.c; fi
echo " PROFILE beam.prof.smp"
 PROFILE beam.prof.smp
rm -f obj/amd64-portbld-freebsd14.0/opt/smp/erl*.profraw
set -e;
LLVM_PROFILE_FILE="obj/amd64-portbld-freebsd14.0/opt/smp/erlc-%m.profraw" \
          ERL_FLAGS="-emu_type prof +S 1" erlc -W  -DPGO \
  -o obj/amd64-portbld-freebsd14.0/opt/smp test/estone_SUITE.erl >
obj/amd64-portbld-freebsd14.0/opt/smp/PROFILE_LOG

after which it does a test run, and uses llvm-profdata to merge the profiling
data into beam_emu_pu.o:

llvm-profdata merge -output
obj/amd64-portbld-freebsd14.0/opt/smp/default.profdata
obj/amd64-portbld-freebsd14.0/opt/smp/*.profraw
cc -fprofile-instr-use=obj/amd64-portbld-freebsd14.0/opt/smp/default.profdata 
-Werror=undef -Werror=implicit -Werror=return-type   -O3 -fomit-frame-pointer
-pipe  -fno-omit-frame-pointer -DMAP_N
ORESERVE=0 -fstack-protector-strong -fno-strict-aliasing 
-I/wrkdirs/usr/ports/lang/erlang/work/otp-OTP-21.3.8.24/erts/amd64-portbld-freebsd14.0
   -DHAVE_CONFIG_H -Wall -Wstrict-prototypes -Wmis
sing-prototypes -Wdeclaration-after-statement -DUSE_THREADS -D_THREAD_SAFE
-D_REENTRANT -DPOSIX_THREADS   -Iamd64-portbld-freebsd14.0/opt/smp -Ibeam
-Isys/unix -Isys/common -Iamd64-portbld-freebs
d14.0 -Ipcre -Ihipe -I../include -I../include/amd64-portbld-freebsd14.0
-I../include/internal -I../include/internal/amd64-portbld-freebsd14.0 -c
beam/beam_emu.c -o obj/amd64-portbld-freebsd14.0/o
pt/smp/beam_emu_pu.o

Later, it runs dtrace over all the collected objects, and this dies:

dtrace -G -C -Ibeam \
  -s beam/erlang_dtrace.d \
  -o obj/amd64-portbld-freebsd14.0/opt/smp/erlang_pu_dtrace.o
  ... long list of objects ...
dtrace: failed to link script beam/erlang_dtrace.d: an error was encountered
while processing obj/amd64-portbld-freebsd14.0/opt/smp/beam_emu_pu.o
gmake[5]: *** [amd64-portbld-freebsd14.0/Makefile:1005:
obj/amd64-portbld-freebsd14.0/opt/smp/erlang_pu_dtrace.o] Error 1
gmake[5]: Leaving directory
'/wrkdirs/usr/ports/lang/erlang/work/otp-OTP-21.3.8.24/erts/emulator'

Something in beam_emu_pu.o (emitted by clang or llvm 13) is tripping up dtrace,
but I have very little knowledge about dtrace so I need help here. :)

Now some other erlang runtimes such as lang/erlang-runtime24 *do* build
successfully with clang 13, but this is only because upstream disabled the PGO
feature, as a side effect of https://github.com/erlang/otp/commit/b165524c732
("erts: Implement the BeamAsm JIT"): 

--- a/erts/configure.in
+++ b/erts/configure.in
...
@@ -704,6 +719,9 @@ else
   fi
 fi

+dnl Disable pgo for now
+USE_PGO=false
+
 AC_SUBST(USE_PGO)
 AC_SUBST(PROFILE_COMPILER)

(This took a *lot* of time to bisect due to erlang's afwul non-linear
history... :)

So, I'm unsure why upstream disabled this "for now", as it has been disabled
for more than a year. We could work around the other erlang failures by also
disabling PGO there, or with help from someone knowledgeable about dtrace, try
to get to the bottom of why dtrace dies on clang 13 produced PGO object files.

-- 
You are receiving this mail because:
You are the assignee for the bug.