Compilation failure with WITH_DTRACE_TESTS on mips/mips and powerpc/powerpc; it's trying to compile a host object on mips/powerpc because MAKEOBJDIRPREFIX is incorrect

[NGie Cooper]

> On Oct 31, 2015, at 14:37, NGie Cooper <yaneurabeya at gmail.com> wrote:
> 
> 
>> On Oct 30, 2015, at 23:51, NGie Cooper <yaneurabeya at gmail.com> wrote:
>> 
>> 
>>> On Oct 30, 2015, at 16:43, Simon J. Gerraty <sjg at juniper.net> wrote:
>>> 
>>> NGie Cooper <yaneurabeya at gmail.com> wrote:
>>>> 	I tried doing buildworld on powerpc/powerpc with -DWITH_DTRACE_TESTS and I ran into this linker issue below. I have no idea (yet) why it’s trying to compile an x64 object when I specify powerpc/powerpc — and more importantly, why is the object not being put in obj.powerpc?
>>>> 	I ran into the same issue on ref11-amd64.freebsd.org when I ran “make tinderbox".
>>> 
>>> Is it possible that the file is left over from a previous build (of amd64?)
>>> 
>>> Does your log show it being built?
>>> 
>>> 
>>>> dtrace -C -x nolibs -G -o usdt.o -s /usr/src/svn/cddl/contrib/opensolaris/cmd/dtrace/test/tst/common/json/usdt.d tst.usdt.o
>>>> dtrace: failed to link script /usr/src/svn/cddl/contrib/opensolaris/cmd/dtrace/test/tst/common/json/usdt.d: incorrect ELF machine type for object file: tst.usdt.o
>>>> *** Error code 1
>>>> $ find /usr/obj/usr/src/svn/ -name tst.usdt.o
>>>> /usr/obj/usr/src/svn/cddl/usr.sbin/dtrace/tests/common/json/tst.usdt.o
>>>> $ file /usr/obj/usr/src/svn/cddl/usr.sbin/dtrace/tests/common/json/tst.usdt.o
>>>> /usr/obj/usr/src/svn/cddl/usr.sbin/dtrace/tests/common/json/tst.usdt.o: ELF 64-bit LSB relocatable, x86-64, version 1 (FreeBSD), not stripped
>> 
>> Still running into issues on ref11-amd64:
>> 
>> cc1: error: unrecognized command line option "-m64"
>> dtrace: failed to compile script /scratch/tmp/ngie/svn/cddl/contrib/opensolaris/cmd/dtrace/test/tst/common/json/usdt.d: Preprocessor failed to process input program
>> --- usdt.h ---
>> *** [usdt.h] Error code 1
>> 
>> Let’s see what happens if I use make buildenv
> 
> …
> 
> Deleting the lines that pass -m32/-m64 gets me back to the same point I was at before:
> 
> dtrace: failed to link script /scratch/tmp/ngie/svn/cddl/contrib/opensolaris/cmd/dtrace/test/tst/common/json/usdt.d: incorrect ELF machine type for object file: tst.usdt.o
> --- usdt.o ---
> *** [usdt.o] Error code 1
> 
> I’ll need to check .PATH, but I think it’s picking up the host copy by accident:
> 
> $ find ../obj/ -name  tst.usdt.o | xargs file
> ../obj/arm.armv6hf/scratch/tmp/ngie/svn/cddl/usr.sbin/dtrace/tests/common/json/tst.usdt.o:       ELF 32-bit LSB relocatable, ARM, EABI5 version 1 (FreeBSD), not stripped
> ../obj/arm.arm/scratch/tmp/ngie/svn/cddl/usr.sbin/dtrace/tests/common/json/tst.usdt.o:           ELF 32-bit LSB relocatable, ARM, EABI5 version 1 (FreeBSD), not stripped
> ../obj/arm.armeb/scratch/tmp/ngie/svn/cddl/usr.sbin/dtrace/tests/common/json/tst.usdt.o:         ELF 32-bit MSB relocatable, ARM, EABI5 version 1 (FreeBSD), not stripped
> ../obj/powerpc.powerpc/scratch/tmp/ngie/svn/cddl/usr.sbin/dtrace/tests/common/json/tst.usdt.o:   ELF 32-bit MSB relocatable, PowerPC or cisco 4500, version 1 (FreeBSD), not stripped
> ../obj/arm.armv6/scratch/tmp/ngie/svn/cddl/usr.sbin/dtrace/tests/common/json/tst.usdt.o:         ELF 32-bit LSB relocatable, ARM, EABI5 version 1 (FreeBSD), not stripped
> ../obj/scratch/tmp/ngie/svn/cddl/usr.sbin/dtrace/tests/common/json/tst.usdt.o:                   ELF 64-bit LSB relocatable, x86-64, version 1 (FreeBSD), not stripped
> ../obj/i386.i386/scratch/tmp/ngie/svn/cddl/usr.sbin/dtrace/tests/common/json/tst.usdt.o:         ELF 32-bit LSB relocatable, Intel 80386, version 1 (FreeBSD), not stripped
> ../obj/pc98.i386/scratch/tmp/ngie/svn/cddl/usr.sbin/dtrace/tests/common/json/tst.usdt.o:         ELF 32-bit LSB relocatable, Intel 80386, version 1 (FreeBSD), not stripped
> ../obj/powerpc.powerpc64/scratch/tmp/ngie/svn/cddl/usr.sbin/dtrace/tests/common/json/tst.usdt.o: ELF 64-bit MSB relocatable, 64-bit PowerPC or cisco 7500, version 1 (FreeBSD), not stripped
> ../obj/arm64.aarch64/scratch/tmp/ngie/svn/cddl/usr.sbin/dtrace/tests/common/json/tst.usdt.o:     ELF 64-bit LSB relocatable, ARM aarch64, version 1 (FreeBSD), not stripped

And here’s the real issue — .PATH is completely broken when TARGET/TARGET_ARCH are specified as explicit values:

$ env MAKEOBJDIRPREFIX=/scratch/tmp/ngie/obj/ make buildenv TARGET=powerpc TARGET_ARCH=powerpc
Entering world for powerpc:powerpc
$ cd cddl/usr.sbin/dtrace/tests/common/json
$ make -V.OBJDIR
/scratch/tmp/ngie/obj//powerpc.powerpc/scratch/tmp/ngie/svn/cddl/usr.sbin/dtrace/tests/common/json
$ make -VMAKEOBJDIRPREFIX
/scratch/tmp/ngie/obj//powerpc.powerpc
$ make depend
(cd /scratch/tmp/ngie/svn/cddl/usr.sbin/dtrace/tests/common/json &&  DEPENDFILE=.depend.tst.usdt.exe  NO_SUBDIR=1 make -f /scratch/tmp/ngie/svn/cddl/usr.sbin/dtrace/tests/common/json/Makefile _RECURSING_PROGS=  PROG=tst.usdt.exe  depend)
$ make all
(cd /scratch/tmp/ngie/svn/cddl/usr.sbin/dtrace/tests/common/json &&  DEPENDFILE=.depend.tst.usdt.exe  NO_SUBDIR=1 make -f /scratch/tmp/ngie/svn/cddl/usr.sbin/dtrace/tests/common/json/Makefile _RECURSING_PROGS=  PROG=tst.usdt.exe )
dtrace -C -x nolibs -G -o usdt.o -s /scratch/tmp/ngie/svn/cddl/contrib/opensolaris/cmd/dtrace/test/tst/common/json/usdt.d tst.usdt.o
dtrace: failed to link script /scratch/tmp/ngie/svn/cddl/contrib/opensolaris/cmd/dtrace/test/tst/common/json/usdt.d: incorrect ELF machine type for object file: tst.usdt.o
*** Error code 1

Stop.
make[3]: stopped in /scratch/tmp/ngie/svn/cddl/usr.sbin/dtrace/tests/common/json
*** Error code 1

Stop.
make[2]: stopped in /scratch/tmp/ngie/svn/cddl/usr.sbin/dtrace/tests/common/json
$ make -V.PATH
. /scratch/tmp/ngie/svn/cddl/usr.sbin/dtrace/tests/common/json /scratch/tmp/ngie/svn/cddl/contrib/opensolaris/cmd/dtrace/test/tst/common/json

Something seems broken with cross-building via `make tinderbox`/`make buildworld`…