Total confusion over toolchain/xdev behavior

Wed Jul 9 00:06:39 UTC 2014

On Mon, Jul 07, 2014 at 07:29:01PM -0600, Warner Losh wrote:
> 
> On Jul 7, 2014, at 5:52 PM, Baptiste Daroussin <bapt at FreeBSD.org> wrote:
> 
> > On Mon, Jul 07, 2014 at 05:27:25PM -0600, Warner Losh wrote:
> >> 
> >> On Jul 7, 2014, at 2:51 PM, Ian Lepore <ian at FreeBSD.org> wrote:
> >> 
> >>> On Sun, 2014-07-06 at 16:07 -0700, Sean Bruno wrote:
> >>>> Objective:  install an xcompile toolchain into a jail for use by
> >>>> poudriere during arm/mips/sparc/power ports pkgs builds.  The build
> >>>> should be possible from a non-root user.
> >>>> 
> >>>> As far as I can tell, the xdev target is completely busted for non-clang
> >>>> arch's right now as it tries to build clang no matter what I do.  Its
> >>>> missing some pretty key documentation to making it work correctly, so a
> >>>> lot of my attempts have been "guess and check" with verbose make.
> >>>> 
> >>>> -----------------------------------------------------------------------
> >>>> Attempt #1:
> >>>> I have been trying non-root xdev builds:
> >>>> MAKEOBJDIRPREFIX=/var/tmp make -s -j8 xdev XDEV=mips XDEV_ARCH=mips
> >>>> -- dies because it tries to compile CLANG.
> >>>> -----------------------------------------------------------------------
> >>>> 
> >>>> Attempt #2:
> >>>> Apply a hack from Baptiste that isn't quite right, but at least gets
> >>>> farther, note the missing variable causing "//usr/mips-freebsd"
> >>>> http://people.freebsd.org/~sbruno/src.ops.mk.diff
> >>>> 
> >>>> ===> gnu/usr.bin/cc/gcov (all)
> >>>> mtree populating //usr/mips-freebsd
> >>>> mkdir: //usr/mips-freebsd: Permission denied
> >>>> *** Error code 1
> >>>> -----------------------------------------------------------------------
> >>>> 
> >>>> Attempt #3: Add XDTP
> >>>> MAKEOBJDIRPREFIX=/var/tmp make -s xdev XDEV=mips XDEV_ARCH=mips
> >>>> XDTP=/var/tmp/mips_cc
> >>>> 
> >>>> Try defining a XDTP=/var/tmp/mips_cc with the above patch applied, get's
> >>>> a bit farther but compile failure in locating critical include files.
> >>>> 
> >>>> ===> gnu/lib/libstdc++ (obj,depend,all,install)
> >>>> In file included from /home/sbruno/bsd/fbsd_head/gnu/lib/libstdc
> >>>> ++/../../../contrib/libstdc++/src/bitmap_allocator.cc:30:
> >>>> /home/sbruno/bsd/fbsd_head/gnu/lib/libstdc++/../../../contrib/libstdc
> >>>> ++/include/ext/bitmap_allocator.h:37:54: error: cstddef: No such file or
> >>>> directory
> >>>> -----------------------------------------------------------------------
> >>>> 
> >>>> Attempt #4: Add the additional XDDESTDIR
> >>>> MAKEOBJDIRPREFIX=/var/tmp make -s xdev XDEV=mips XDEV_ARCH=mips
> >>>> XDTP=/var/tmp/mips_cc XDESTDIR=/var/tmp/mips_dst
> >>>> -- Same results as attempt #3
> >>>> -----------------------------------------------------------------------
> >>>> 
> >>>> Even attempting to do stuff for *clang* enabled architectures bails
> >>>> because its not respecting prefixes:
> >>>> MAKEOBJDIRPREFIX=/var/tmp make -s -j 8 xdev XDEV=arm XDEV_ARCH=armv6
> >>>> -- bails because it tries to:
> >>>> ===> usr.bin/clang/tblgen (all)
> >>>> mtree populating //usr/armv6-freebsd
> >>>> mtree: etc/ntp: Permission denied
> >>>> _xi-cross-tools
> >>>> ===> xdev gnu/usr.bin/binutils (install)
> >>>> ===> gnu/usr.bin/binutils/libiberty (install)
> >>>> ===> gnu/usr.bin/binutils/libbfd (install)
> >>>> ===> gnu/usr.bin/binutils/libopcodes (install)
> >>>> ===> gnu/usr.bin/binutils/libbinutils (install)
> >>>> ===> gnu/usr.bin/binutils/addr2line (install)
> >>>> ===> gnu/usr.bin/binutils/as (install)
> >>>> ===> gnu/usr.bin/binutils/ld (install)
> >>>> install: //usr/armv6-freebsd/usr/bin/ld: Permission denied
> >>>> *** Error code 71
> >>>> 
> >>>> -----------------------------------------------------------------------
> >>>> Adding XDTP and XDDESTDIR results in a little more progress but obvious
> >>>> failures to attempt and install things directly into my host system:
> >>>> 
> >>>> MAKEOBJDIRPREFIX=/var/tmp make -s xdev XDEV=arm XDEV_ARCH=armv6
> >>>> XDDESTDIR=/var/tmp/arm_cc XDTP=/var/tmp/armv6_cc
> >>>> ===> secure/lib/libssh (install)
> >>>> ===> usr.bin/lex/lib (obj,depend,all,install)
> >>>> mkdir: ../../../../usr: Permission denied
> >>>> *** Error code 1
> >>>> 
> >>>> Stop.
> >>>> make[1]: stopped in /home/sbruno/bsd/fbsd_head
> >>>> *** Error code 1
> >>>> 
> >>>> Stop.
> >>>> make: stopped in /home/sbruno/bsd/fbsd_head
> >>> 
> >>> It looks to me like the permission part of the problem is being caused
> >>> by a lack of DESTDIR=.  Without that, it's trying to install to /usr and
> >>> you don't have permission for that.  Maybe the confusion is because the
> >>> xdev target inherently builds-and-installs, unlike most other targets
> >>> that separate those two actions.
> >> 
> >> OK. After some detective work, it looks like libstdc++ needs to be done before libsupc++ is done. I’ve added this dependency in r268377 and was able to do a full xdev build with a clean obj dir:
> >> 
> >> 	rm -rf $HOME/F $MAKEOBJDIRPREFIX/mips-freebsd
> >> 	mkdir $HOME/F
> >> 	make xdev DESTDIR=$HOME/F XDEV=mips XDEV_ARCH=mips WITHOUT_CLANG=t WITHOUT_CLANG_BOOTSTRAP=t WITH_GCC=t WITH_GCC_BOOTSTRAP=t WITH_GNUCXX=t -j 20
> > 
> > We can avoid most of the above by using a patch like the following:
> > http://people.freebsd.org/~bapt/Makefile.inc1.diff
> > Extending the same thing xi-cross-tools and xb-cross-tools (expect the
> > WITH_GNUCXX=t because it it not set in src.opts.mk when it imho should.)
> 
> The patch looks good to my eye. Did you want me to expand it, or do you want to
> do the honors?
> 
> About the rest… Yea, you may be right….  MK_GNUCXX is an odd duck, and that’s
> likely the problem that should be fixed in a different way. It is really an internal
> variable that should be set based on the actual compiler type (possibly with an
> override for the odd-duck pair of clang and libstdc++ which may not be worth
> supporting). It is telling us we’re doing something horribly wrong and we should listen
> to that rather than add another compiler-related kludge to the build system.  I’ll work
> on that bit.
> 
> Also an aside: The horribly long command line was to try to get to the bottom of
> the breakage, not to promote it as the proper way of doing things.

I haven't committed because it isn't complete yet :)
I have updated it to the following:
http://people.freebsd.org/~bapt/Makefile.inc1.diff
but it is not enough as I end up with:

/tmp/tmips//usr/mips64-freebsd/usr/bin/ld: cannot find -lsupc++

So it goes further but it is not yet enough

regards,
Bapt
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 181 bytes
Desc: not available
URL: <http://lists.freebsd.org/pipermail/freebsd-arch/attachments/20140709/19c2db6c/attachment.sig>