Total confusion over toolchain/xdev behavior

Tue Jul 8 01:29:18 UTC 2014

On Jul 7, 2014, at 5:52 PM, Baptiste Daroussin <bapt at FreeBSD.org> wrote:

> On Mon, Jul 07, 2014 at 05:27:25PM -0600, Warner Losh wrote:
>> 
>> On Jul 7, 2014, at 2:51 PM, Ian Lepore <ian at FreeBSD.org> wrote:
>> 
>>> On Sun, 2014-07-06 at 16:07 -0700, Sean Bruno wrote:
>>>> Objective:  install an xcompile toolchain into a jail for use by
>>>> poudriere during arm/mips/sparc/power ports pkgs builds.  The build
>>>> should be possible from a non-root user.
>>>> 
>>>> As far as I can tell, the xdev target is completely busted for non-clang
>>>> arch's right now as it tries to build clang no matter what I do.  Its
>>>> missing some pretty key documentation to making it work correctly, so a
>>>> lot of my attempts have been "guess and check" with verbose make.
>>>> 
>>>> -----------------------------------------------------------------------
>>>> Attempt #1:
>>>> I have been trying non-root xdev builds:
>>>> MAKEOBJDIRPREFIX=/var/tmp make -s -j8 xdev XDEV=mips XDEV_ARCH=mips
>>>> -- dies because it tries to compile CLANG.
>>>> -----------------------------------------------------------------------
>>>> 
>>>> Attempt #2:
>>>> Apply a hack from Baptiste that isn't quite right, but at least gets
>>>> farther, note the missing variable causing "//usr/mips-freebsd"
>>>> http://people.freebsd.org/~sbruno/src.ops.mk.diff
>>>> 
>>>> ===> gnu/usr.bin/cc/gcov (all)
>>>> mtree populating //usr/mips-freebsd
>>>> mkdir: //usr/mips-freebsd: Permission denied
>>>> *** Error code 1
>>>> -----------------------------------------------------------------------
>>>> 
>>>> Attempt #3: Add XDTP
>>>> MAKEOBJDIRPREFIX=/var/tmp make -s xdev XDEV=mips XDEV_ARCH=mips
>>>> XDTP=/var/tmp/mips_cc
>>>> 
>>>> Try defining a XDTP=/var/tmp/mips_cc with the above patch applied, get's
>>>> a bit farther but compile failure in locating critical include files.
>>>> 
>>>> ===> gnu/lib/libstdc++ (obj,depend,all,install)
>>>> In file included from /home/sbruno/bsd/fbsd_head/gnu/lib/libstdc
>>>> ++/../../../contrib/libstdc++/src/bitmap_allocator.cc:30:
>>>> /home/sbruno/bsd/fbsd_head/gnu/lib/libstdc++/../../../contrib/libstdc
>>>> ++/include/ext/bitmap_allocator.h:37:54: error: cstddef: No such file or
>>>> directory
>>>> -----------------------------------------------------------------------
>>>> 
>>>> Attempt #4: Add the additional XDDESTDIR
>>>> MAKEOBJDIRPREFIX=/var/tmp make -s xdev XDEV=mips XDEV_ARCH=mips
>>>> XDTP=/var/tmp/mips_cc XDESTDIR=/var/tmp/mips_dst
>>>> -- Same results as attempt #3
>>>> -----------------------------------------------------------------------
>>>> 
>>>> Even attempting to do stuff for *clang* enabled architectures bails
>>>> because its not respecting prefixes:
>>>> MAKEOBJDIRPREFIX=/var/tmp make -s -j 8 xdev XDEV=arm XDEV_ARCH=armv6
>>>> -- bails because it tries to:
>>>> ===> usr.bin/clang/tblgen (all)
>>>> mtree populating //usr/armv6-freebsd
>>>> mtree: etc/ntp: Permission denied
>>>> _xi-cross-tools
>>>> ===> xdev gnu/usr.bin/binutils (install)
>>>> ===> gnu/usr.bin/binutils/libiberty (install)
>>>> ===> gnu/usr.bin/binutils/libbfd (install)
>>>> ===> gnu/usr.bin/binutils/libopcodes (install)
>>>> ===> gnu/usr.bin/binutils/libbinutils (install)
>>>> ===> gnu/usr.bin/binutils/addr2line (install)
>>>> ===> gnu/usr.bin/binutils/as (install)
>>>> ===> gnu/usr.bin/binutils/ld (install)
>>>> install: //usr/armv6-freebsd/usr/bin/ld: Permission denied
>>>> *** Error code 71
>>>> 
>>>> -----------------------------------------------------------------------
>>>> Adding XDTP and XDDESTDIR results in a little more progress but obvious
>>>> failures to attempt and install things directly into my host system:
>>>> 
>>>> MAKEOBJDIRPREFIX=/var/tmp make -s xdev XDEV=arm XDEV_ARCH=armv6
>>>> XDDESTDIR=/var/tmp/arm_cc XDTP=/var/tmp/armv6_cc
>>>> ===> secure/lib/libssh (install)
>>>> ===> usr.bin/lex/lib (obj,depend,all,install)
>>>> mkdir: ../../../../usr: Permission denied
>>>> *** Error code 1
>>>> 
>>>> Stop.
>>>> make[1]: stopped in /home/sbruno/bsd/fbsd_head
>>>> *** Error code 1
>>>> 
>>>> Stop.
>>>> make: stopped in /home/sbruno/bsd/fbsd_head
>>> 
>>> It looks to me like the permission part of the problem is being caused
>>> by a lack of DESTDIR=.  Without that, it's trying to install to /usr and
>>> you don't have permission for that.  Maybe the confusion is because the
>>> xdev target inherently builds-and-installs, unlike most other targets
>>> that separate those two actions.
>> 
>> OK. After some detective work, it looks like libstdc++ needs to be done before libsupc++ is done. I’ve added this dependency in r268377 and was able to do a full xdev build with a clean obj dir:
>> 
>> 	rm -rf $HOME/F $MAKEOBJDIRPREFIX/mips-freebsd
>> 	mkdir $HOME/F
>> 	make xdev DESTDIR=$HOME/F XDEV=mips XDEV_ARCH=mips WITHOUT_CLANG=t WITHOUT_CLANG_BOOTSTRAP=t WITH_GCC=t WITH_GCC_BOOTSTRAP=t WITH_GNUCXX=t -j 20
> 
> We can avoid most of the above by using a patch like the following:
> http://people.freebsd.org/~bapt/Makefile.inc1.diff
> Extending the same thing xi-cross-tools and xb-cross-tools (expect the
> WITH_GNUCXX=t because it it not set in src.opts.mk when it imho should.)

The patch looks good to my eye. Did you want me to expand it, or do you want to
do the honors?

About the rest… Yea, you may be right….  MK_GNUCXX is an odd duck, and that’s
likely the problem that should be fixed in a different way. It is really an internal
variable that should be set based on the actual compiler type (possibly with an
override for the odd-duck pair of clang and libstdc++ which may not be worth
supporting). It is telling us we’re doing something horribly wrong and we should listen
to that rather than add another compiler-related kludge to the build system.  I’ll work
on that bit.

Also an aside: The horribly long command line was to try to get to the bottom of
the breakage, not to promote it as the proper way of doing things.

Warner
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 842 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://lists.freebsd.org/pipermail/freebsd-arch/attachments/20140707/868320c8/attachment.sig>