Total confusion over toolchain/xdev behavior

Wed Jul 9 06:09:43 UTC 2014

On Tue, Jul 08, 2014 at 09:48:45PM -0600, Warner Losh wrote:
> 
> On Jul 8, 2014, at 6:06 PM, Baptiste Daroussin <bapt at FreeBSD.org> wrote:
> 
> > On Mon, Jul 07, 2014 at 07:29:01PM -0600, Warner Losh wrote:
> >> 
> >> On Jul 7, 2014, at 5:52 PM, Baptiste Daroussin <bapt at FreeBSD.org> wrote:
> >> 
> >>> On Mon, Jul 07, 2014 at 05:27:25PM -0600, Warner Losh wrote:
> >>>> 
> >>>> On Jul 7, 2014, at 2:51 PM, Ian Lepore <ian at FreeBSD.org> wrote:
> >>>> 
> >>>>> On Sun, 2014-07-06 at 16:07 -0700, Sean Bruno wrote:
> >>>>>> Objective:  install an xcompile toolchain into a jail for use by
> >>>>>> poudriere during arm/mips/sparc/power ports pkgs builds.  The build
> >>>>>> should be possible from a non-root user.
> >>>>>> 
> >>>>>> As far as I can tell, the xdev target is completely busted for non-clang
> >>>>>> arch's right now as it tries to build clang no matter what I do.  Its
> >>>>>> missing some pretty key documentation to making it work correctly, so a
> >>>>>> lot of my attempts have been "guess and check" with verbose make.
> >>>>>> 
> >>>>>> -----------------------------------------------------------------------
> >>>>>> Attempt #1:
> >>>>>> I have been trying non-root xdev builds:
> >>>>>> MAKEOBJDIRPREFIX=/var/tmp make -s -j8 xdev XDEV=mips XDEV_ARCH=mips
> >>>>>> -- dies because it tries to compile CLANG.
> >>>>>> -----------------------------------------------------------------------
> >>>>>> 
> >>>>>> Attempt #2:
> >>>>>> Apply a hack from Baptiste that isn't quite right, but at least gets
> >>>>>> farther, note the missing variable causing "//usr/mips-freebsd"
> >>>>>> http://people.freebsd.org/~sbruno/src.ops.mk.diff
> >>>>>> 
> >>>>>> ===> gnu/usr.bin/cc/gcov (all)
> >>>>>> mtree populating //usr/mips-freebsd
> >>>>>> mkdir: //usr/mips-freebsd: Permission denied
> >>>>>> *** Error code 1
> >>>>>> -----------------------------------------------------------------------
> >>>>>> 
> >>>>>> Attempt #3: Add XDTP
> >>>>>> MAKEOBJDIRPREFIX=/var/tmp make -s xdev XDEV=mips XDEV_ARCH=mips
> >>>>>> XDTP=/var/tmp/mips_cc
> >>>>>> 
> >>>>>> Try defining a XDTP=/var/tmp/mips_cc with the above patch applied, get's
> >>>>>> a bit farther but compile failure in locating critical include files.
> >>>>>> 
> >>>>>> ===> gnu/lib/libstdc++ (obj,depend,all,install)
> >>>>>> In file included from /home/sbruno/bsd/fbsd_head/gnu/lib/libstdc
> >>>>>> ++/../../../contrib/libstdc++/src/bitmap_allocator.cc:30:
> >>>>>> /home/sbruno/bsd/fbsd_head/gnu/lib/libstdc++/../../../contrib/libstdc
> >>>>>> ++/include/ext/bitmap_allocator.h:37:54: error: cstddef: No such file or
> >>>>>> directory
> >>>>>> -----------------------------------------------------------------------
> >>>>>> 
> >>>>>> Attempt #4: Add the additional XDDESTDIR
> >>>>>> MAKEOBJDIRPREFIX=/var/tmp make -s xdev XDEV=mips XDEV_ARCH=mips
> >>>>>> XDTP=/var/tmp/mips_cc XDESTDIR=/var/tmp/mips_dst
> >>>>>> -- Same results as attempt #3
> >>>>>> -----------------------------------------------------------------------
> >>>>>> 
> >>>>>> Even attempting to do stuff for *clang* enabled architectures bails
> >>>>>> because its not respecting prefixes:
> >>>>>> MAKEOBJDIRPREFIX=/var/tmp make -s -j 8 xdev XDEV=arm XDEV_ARCH=armv6
> >>>>>> -- bails because it tries to:
> >>>>>> ===> usr.bin/clang/tblgen (all)
> >>>>>> mtree populating //usr/armv6-freebsd
> >>>>>> mtree: etc/ntp: Permission denied
> >>>>>> _xi-cross-tools
> >>>>>> ===> xdev gnu/usr.bin/binutils (install)
> >>>>>> ===> gnu/usr.bin/binutils/libiberty (install)
> >>>>>> ===> gnu/usr.bin/binutils/libbfd (install)
> >>>>>> ===> gnu/usr.bin/binutils/libopcodes (install)
> >>>>>> ===> gnu/usr.bin/binutils/libbinutils (install)
> >>>>>> ===> gnu/usr.bin/binutils/addr2line (install)
> >>>>>> ===> gnu/usr.bin/binutils/as (install)
> >>>>>> ===> gnu/usr.bin/binutils/ld (install)
> >>>>>> install: //usr/armv6-freebsd/usr/bin/ld: Permission denied
> >>>>>> *** Error code 71
> >>>>>> 
> >>>>>> -----------------------------------------------------------------------
> >>>>>> Adding XDTP and XDDESTDIR results in a little more progress but obvious
> >>>>>> failures to attempt and install things directly into my host system:
> >>>>>> 
> >>>>>> MAKEOBJDIRPREFIX=/var/tmp make -s xdev XDEV=arm XDEV_ARCH=armv6
> >>>>>> XDDESTDIR=/var/tmp/arm_cc XDTP=/var/tmp/armv6_cc
> >>>>>> ===> secure/lib/libssh (install)
> >>>>>> ===> usr.bin/lex/lib (obj,depend,all,install)
> >>>>>> mkdir: ../../../../usr: Permission denied
> >>>>>> *** Error code 1
> >>>>>> 
> >>>>>> Stop.
> >>>>>> make[1]: stopped in /home/sbruno/bsd/fbsd_head
> >>>>>> *** Error code 1
> >>>>>> 
> >>>>>> Stop.
> >>>>>> make: stopped in /home/sbruno/bsd/fbsd_head
> >>>>> 
> >>>>> It looks to me like the permission part of the problem is being caused
> >>>>> by a lack of DESTDIR=.  Without that, it's trying to install to /usr and
> >>>>> you don't have permission for that.  Maybe the confusion is because the
> >>>>> xdev target inherently builds-and-installs, unlike most other targets
> >>>>> that separate those two actions.
> >>>> 
> >>>> OK. After some detective work, it looks like libstdc++ needs to be done before libsupc++ is done. I’ve added this dependency in r268377 and was able to do a full xdev build with a clean obj dir:
> >>>> 
> >>>> 	rm -rf $HOME/F $MAKEOBJDIRPREFIX/mips-freebsd
> >>>> 	mkdir $HOME/F
> >>>> 	make xdev DESTDIR=$HOME/F XDEV=mips XDEV_ARCH=mips WITHOUT_CLANG=t WITHOUT_CLANG_BOOTSTRAP=t WITH_GCC=t WITH_GCC_BOOTSTRAP=t WITH_GNUCXX=t -j 20
> >>> 
> >>> We can avoid most of the above by using a patch like the following:
> >>> http://people.freebsd.org/~bapt/Makefile.inc1.diff
> >>> Extending the same thing xi-cross-tools and xb-cross-tools (expect the
> >>> WITH_GNUCXX=t because it it not set in src.opts.mk when it imho should.)
> >> 
> >> The patch looks good to my eye. Did you want me to expand it, or do you want to
> >> do the honors?
> >> 
> >> About the rest… Yea, you may be right….  MK_GNUCXX is an odd duck, and that’s
> >> likely the problem that should be fixed in a different way. It is really an internal
> >> variable that should be set based on the actual compiler type (possibly with an
> >> override for the odd-duck pair of clang and libstdc++ which may not be worth
> >> supporting). It is telling us we’re doing something horribly wrong and we should listen
> >> to that rather than add another compiler-related kludge to the build system.  I’ll work
> >> on that bit.
> >> 
> >> Also an aside: The horribly long command line was to try to get to the bottom of
> >> the breakage, not to promote it as the proper way of doing things.
> > 
> > I haven't committed because it isn't complete yet :)
> > I have updated it to the following:
> > http://people.freebsd.org/~bapt/Makefile.inc1.diff
> > but it is not enough as I end up with:
> > 
> > /tmp/tmips//usr/mips64-freebsd/usr/bin/ld: cannot find -lsupc++
> > 
> > So it goes further but it is not yet enough
> 
> I actually think xdev and a couple others should move to Makefile, and the sub targets stay in Makefile.inc and be invoked with the right TARGET and TARGET_ARCH so that we get the right defaults.
> 
I 100% agree on that :)

regards,
Bapt
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 181 bytes
Desc: not available
URL: <http://lists.freebsd.org/pipermail/freebsd-arch/attachments/20140709/4d8ed3f8/attachment.sig>