Total confusion over toolchain/xdev behavior

[Warner Losh]

On Jul 7, 2014, at 2:51 PM, Ian Lepore <ian at FreeBSD.org> wrote:

> On Sun, 2014-07-06 at 16:07 -0700, Sean Bruno wrote:
>> Objective:  install an xcompile toolchain into a jail for use by
>> poudriere during arm/mips/sparc/power ports pkgs builds.  The build
>> should be possible from a non-root user.
>> 
>> As far as I can tell, the xdev target is completely busted for non-clang
>> arch's right now as it tries to build clang no matter what I do.  Its
>> missing some pretty key documentation to making it work correctly, so a
>> lot of my attempts have been "guess and check" with verbose make.
>> 
>> -----------------------------------------------------------------------
>> Attempt #1:
>> I have been trying non-root xdev builds:
>> MAKEOBJDIRPREFIX=/var/tmp make -s -j8 xdev XDEV=mips XDEV_ARCH=mips
>> -- dies because it tries to compile CLANG.
>> -----------------------------------------------------------------------
>> 
>> Attempt #2:
>> Apply a hack from Baptiste that isn't quite right, but at least gets
>> farther, note the missing variable causing "//usr/mips-freebsd"
>> http://people.freebsd.org/~sbruno/src.ops.mk.diff
>> 
>> ===> gnu/usr.bin/cc/gcov (all)
>> mtree populating //usr/mips-freebsd
>> mkdir: //usr/mips-freebsd: Permission denied
>> *** Error code 1
>> -----------------------------------------------------------------------
>> 
>> Attempt #3: Add XDTP
>> MAKEOBJDIRPREFIX=/var/tmp make -s xdev XDEV=mips XDEV_ARCH=mips
>> XDTP=/var/tmp/mips_cc
>> 
>> Try defining a XDTP=/var/tmp/mips_cc with the above patch applied, get's
>> a bit farther but compile failure in locating critical include files.
>> 
>> ===> gnu/lib/libstdc++ (obj,depend,all,install)
>> In file included from /home/sbruno/bsd/fbsd_head/gnu/lib/libstdc
>> ++/../../../contrib/libstdc++/src/bitmap_allocator.cc:30:
>> /home/sbruno/bsd/fbsd_head/gnu/lib/libstdc++/../../../contrib/libstdc
>> ++/include/ext/bitmap_allocator.h:37:54: error: cstddef: No such file or
>> directory
>> -----------------------------------------------------------------------
>> 
>> Attempt #4: Add the additional XDDESTDIR
>> MAKEOBJDIRPREFIX=/var/tmp make -s xdev XDEV=mips XDEV_ARCH=mips
>> XDTP=/var/tmp/mips_cc XDESTDIR=/var/tmp/mips_dst
>> -- Same results as attempt #3
>> -----------------------------------------------------------------------
>> 
>> Even attempting to do stuff for *clang* enabled architectures bails
>> because its not respecting prefixes:
>> MAKEOBJDIRPREFIX=/var/tmp make -s -j 8 xdev XDEV=arm XDEV_ARCH=armv6
>> -- bails because it tries to:
>> ===> usr.bin/clang/tblgen (all)
>> mtree populating //usr/armv6-freebsd
>> mtree: etc/ntp: Permission denied
>> _xi-cross-tools
>> ===> xdev gnu/usr.bin/binutils (install)
>> ===> gnu/usr.bin/binutils/libiberty (install)
>> ===> gnu/usr.bin/binutils/libbfd (install)
>> ===> gnu/usr.bin/binutils/libopcodes (install)
>> ===> gnu/usr.bin/binutils/libbinutils (install)
>> ===> gnu/usr.bin/binutils/addr2line (install)
>> ===> gnu/usr.bin/binutils/as (install)
>> ===> gnu/usr.bin/binutils/ld (install)
>> install: //usr/armv6-freebsd/usr/bin/ld: Permission denied
>> *** Error code 71
>> 
>> -----------------------------------------------------------------------
>> Adding XDTP and XDDESTDIR results in a little more progress but obvious
>> failures to attempt and install things directly into my host system:
>> 
>> MAKEOBJDIRPREFIX=/var/tmp make -s xdev XDEV=arm XDEV_ARCH=armv6
>> XDDESTDIR=/var/tmp/arm_cc XDTP=/var/tmp/armv6_cc
>> ===> secure/lib/libssh (install)
>> ===> usr.bin/lex/lib (obj,depend,all,install)
>> mkdir: ../../../../usr: Permission denied
>> *** Error code 1
>> 
>> Stop.
>> make[1]: stopped in /home/sbruno/bsd/fbsd_head
>> *** Error code 1
>> 
>> Stop.
>> make: stopped in /home/sbruno/bsd/fbsd_head
> 
> It looks to me like the permission part of the problem is being caused
> by a lack of DESTDIR=.  Without that, it's trying to install to /usr and
> you don't have permission for that.  Maybe the confusion is because the
> xdev target inherently builds-and-installs, unlike most other targets
> that separate those two actions.

OK. After some detective work, it looks like libstdc++ needs to be done before libsupc++ is done. I’ve added this dependency in r268377 and was able to do a full xdev build with a clean obj dir:

	rm -rf $HOME/F $MAKEOBJDIRPREFIX/mips-freebsd
	mkdir $HOME/F
	make xdev DESTDIR=$HOME/F XDEV=mips XDEV_ARCH=mips WITHOUT_CLANG=t WITHOUT_CLANG_BOOTSTRAP=t WITH_GCC=t WITH_GCC_BOOTSTRAP=t WITH_GNUCXX=t -j 20

Sean, can you see if this works for you now? And sorry about the cumbersome options. Those are in line to get fixed, but I wanna fix the build-in-tree issues first (which, alas, are harder than you’d think). I suspect that bapt’s src.opts.mk diffs are likely good candidates to be used committed too, but the above works w/o it. Also, I just built the binaries, I didn’t see if they worked.

Warner
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 842 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://lists.freebsd.org/pipermail/freebsd-arch/attachments/20140707/988189c3/attachment.sig>