Cross Compiling of ports Makefiles.

[Michael Vale]

Hi, 

For those of you who are aware I’ve been implementing a complete cross-compiling series of functions to ports makefiles.

I had a good 3+ week break since my last email with a patch to show, and I’ve totally re-written it and have started from scratch.  Not including any of Ray’s Zrouter code either.

While it’s still a work in progress, i have outlined the entire system to produce target installs into the same staging directory as a bsd system ready to be flashed onto NAND for embedded, complete with pkg registry and ldconfig, everything has been thought of.   - The reason I have chosen this method for the ports to be installed into a tree is so they can be compliled after build/install kernel/world and be combined into one firmware image seemlessly.  Some ports won’t just be optional applications for future embedded firmware images, they’ll be an integral part of it.  The goal here is to be able to build complete firmware images in one fowl swoop.  Perhaps beyond the scope most of you out there but I may wish to pick and choose exclude required parts of the BSD system and replace them with the busybox port and replace libc with google’s Bionic, uClibc or even musl.  This cannot be achieved currently with the likes of tinderbox and pourdiere

It will still be possible to build packages though.

Due to the nature of cross building first i’ll lay out the options and then tell you which one I am implementing first as there are reasons for having different build-enviornments/toolchains.

Ok, firstly I was going to give you all detail of all possible cross-compiling scenarios as I outline them. but I’ll have you know it’s much of a muchness, there is the pros and cons to each and every different step, the one i’m about to put to you now is the most feature complete and quickest to implement.  That doesn’t mean building without a DESTDIR JAIL in the future and just using the build system and it’s tools without a new toolchain doesn’t make sense (sometimes it does!) and that i’m not going to do it or that I’m not going to do a full '’Canadian Cross’.

Ultimately as a goal the minimal command do invoke cross compliation is TARGET(_ARCH)=${ARCH} make.

This could go on for hours, so after just deleted to extra paragraphs, i’m going to summerise.

first we check for CLANG (as the x-compiler) or if we need to install xdev (bsd make of gcc compiled for target arch).
(ok so some of this wont be in Makefile order (upside down and back to front), but im just spitting it out as it comes)
if GNU configure is used, it usually pretty good at detecting the compilers executable path from the TARGET triple alone, for worse case scenario also set ${CC}’s path at the beginning of global env ${PATH} to override any subsequent.

pre-chroot: is mostly used to declare global env variables to keep the build from failing and making sure the install will complete.

do-chroot: and we have to firstly install and BUILD_DEPENDS, remember these can be libraries too and they have to be built with the build machines usual stuff and installed in their usual place (lucky we are using a CHROOTED JAIL here! we could easy make a mess otherwise) remembering sometimes some depends can be both a BUILD dep AND a RUN dep to the TARGET.  That’s okay, they should always be declared as correctly and never have to cross-compile a BUILD depend.  However a BUILD depend can be build twice, (once for the build system) and again (as a TARGET) for the TARGET as a RUN depend for the TARGET.

The beauty of doing this work is we can now treat the lib and run depends more suitably.  During this process we can strip the libs, exclude the headers and change the directory structure to one, save on inodes, and second pkg register, libtool and ld require the files are installed into the root tree correctly in order for them to build valid databases and register them. Now, BUILD/HOST system has already had it’s tail cut off by DESTDIR.  Now there is plenty of ways we can install everything into a valid sub-directory and have DESTDIR still considered ROOT and PREFIX or LOCALDIR doesn’t have some obscure prepending directory that doesn’t exist in the CROSS_STAGING_ROOT.  Some ways include adding a variable in bsd.lib.mk and in every single one of make’s install targets between ${DESTDIR} and ${LOCALBASE} or ${PREFIX}.  And we could include if statements for cross, this would leave it at that and we could go ahead and simply install into a sub-directory before pkg, ldconfig and firmware image packing occurs, but I’d rather keep all cross-building to bsd.cross.mk and include it in bsd.port.mk and instead within DESTDIR do-chroot: re-define ${DESTDIR} as ${_bldroot}${DESTDIR} and all TARGET_LIBS, RUN_DEPENDS and TARGET install in a CHROOTED=no chroot.

Doing the same thing could also prevent the need for a DESTDIR JAIL install at all and just use the real build machine’s build env, rather than a jail.  Regardless.  We still have to install these targets and their DESTDIR is skewed.  There is a few options,

One is to have a MAKEOBJDIRPREFIX like option, and redefine every target’s DESTDIR ${makeobjDESTDIR} before running do-install.  Now i’ve yet to complete this stage, but I believe this is the way to do it.

There are other options but they aren’t as elegant/will make baby jesus cry.

Now the install of these targets won’t require a chroot.  A chroot could be done, and that would be okay for one port.  but if there is already a cross compiled system in there ready for flashing to disk, theres no way to chroot without moving files temporarially form the existing target system and copying or building programs like /bin/sh that will execute on the build machine and allow chroot to run.



We can patch/sed PLIST files, for pkg register to work, patch/sed/edit ldconfig’s db, and some other steps.  But I don’t like that idea.


that’s why I’m opting with the other option and that is to create some INSTALL_DEPENDS or CROSS_COMPILING_CHROOT_INSTALL_DEPENDS, if you will.  just /bin/sh and another few 
build TARGET port in jailed DESTDIR/CHROOTED=yes.

this is achieved by installing all build dependencies first...


Sorry, I’m too tired to continue on any further!

I wanted to wait until the initial plan works, shoot an email off then get into the good stuff.  But it’s taking me longer than I thought even just to describe all the processes.

I didn’t want to submit half-baked Makefiles that don’t work, but I can only write about half of one anyway haha!

Anyway, I’m going to spend some time working on them in the next few days, so please expect an update.