Cross platform building best practices (building 6 on 7)

[Alexander Sack]

On Fri, Jun 20, 2008 at 6:02 PM, Julian Elischer <julian at elischer.org> wrote:
> Alexander Sack wrote:
>>
>> On Fri, Jun 20, 2008 at 5:14 PM, Julian Elischer <julian at elischer.org>
>> wrote:
>>>
>>> Alexander Sack wrote:
>>>>
>>>> On Thu, Jun 19, 2008 at 11:22 AM, Garrett Cooper <yanefbsd at gmail.com>
>>>> wrote:
>>>>>
>>>>> On Thu, Jun 19, 2008 at 6:29 AM, Alexander Sack <pisymbol at gmail.com>
>>>>> wrote:
>>>>>>
>>>>>> Hello Folks:
>>>>>>
>>>>>> I've done a lot of Googling and scouring the lists about this
>>>>>> particular subject so I apologize for rehashing it.  However, I'm
>>>>>> still confused on what's the best way to perform BSD cross platform
>>>>>> builds.  Ideally what I want to have is an environment whereby I can
>>>>>> build a 6.1-RELEASE tree on a 7.0-RELEASE box.  I thought originally I
>>>>>> could check out a 6.1 release version, perform make world, and then
>>>>>> use the output of that build as either a basis for a jail or a
>>>>>> toolchain.  However, as noted by previous threads, 6.x doesn't build
>>>>>> on a 7.x due to gcc4/binutils compatibility issues (please correct me
>>>>>> if I'm wrong).  I then thought I could potentially download a patched
>>>>>> binutils, copy it into src/contrib/binutils and that would potentially
>>>>>> fix it.  No dice (and I'm still debugging why since this binutils
>>>>>> package DOES build outside of the make world infrastructure without
>>>>>> issue, this very well could be pilot error on my part since I didn't
>>>>>> update the VERSION string and didn't trim the source files as per the
>>>>>> FreeBSD-deleteList etc.).
>>>>>>
>>>>>> I THEN thought if I build/install a gcc-3.x/bintuils toolchain I could
>>>>>> complie a 6.x on a 7.x machine.  Well I haven't done that yet since at
>>>>>> this point I believe I'm diverged from the path of FreeBSD build
>>>>>> enlightenment!  Moreover, if would be NICE if I could bootstrap the
>>>>>> normal dev tools from the exiting make world build tree.  I'm not yet
>>>>>> ready for a lot of hackery on the build tree without asking around.
>>>>>> :D!
>>>>>>
>>>>>> Does anyone due cross-platform builds (without host virtualization)?
>>>>>>
>>>>>> Thanks!
>>>>>>
>>>>>> -aps
>>>>>
>>>>> (I'll stick to just hackers@ because I don't want to pollute
>>>>> questions@ unnecessarily)
>>>>
>>>> Sorry I felt really bad actually cc'ing questions its just that my
>>>> last groking produced many threads in freebsd-questions as opposed to
>>>> hackers.  I'll try to be more attentive to my posts (I have a habit
>>>> cc'ing multiple forums because sometimes they apply but questions is
>>>> for normal troubleshooting, not cross-platform build issues!).
>>>>
>>>>> You touched on an important point. There were some code quality issues
>>>>> (I think) with 6.x that were resolved moving to 7.x, which caused
>>>>> gcc-4.2.x to barf.
>>>>
>>>> Probably but I'm not trying to point fingers!  :D!
>>>>
>>>>> gcc-4.2.x requires a newer version of binutils, just because (for API
>>>>> / usage compatibility).
>>>>
>>>> Yea understood.  To be honest, this isn't documented very readily.  I
>>>> first thought it was pilot error on me, then I decided to take a look
>>>> at what failed to compile (I believe it was an innocent extern).  And
>>>> then got lost in gcc/binutils hell.  Luckily I've smelled this problem
>>>> before and after some research confirmed by suspicion.
>>>>
>>>>> What you should probably do is create a jail then do your development
>>>>> for 6.x in a jail, 7.x in another, and (if you're bold enough ;)...)
>>>>> do 8.x development in yet a third. Jail's are a much better way to
>>>>> isolate things such that you don't have to worry about toolchain
>>>>> issues like these and are able to setup a sourcebase as the devs
>>>>> intended it (for the most part; you may run into issues with sysctls
>>>>> and virtual kernel stuff like that, but cest la vie... there isn't a
>>>>> better way I know of than that outside of running a VM).
>>>>
>>>> I figured you were going ot say that Garrett.  Well OK, but I still
>>>> need to bootstrap my dev environment for 6.x development on 7.x.
>>>> Since binutils compatibility makes my 6.x make world barf on 7.x,
>>>> where should I go?  I HAVE not parsed through a lot of the build
>>>> infrastructure yet but it would seem to be IF make world bootstraps
>>>> the world including the development tools, why can't I update
>>>> binutils/gcc inplace and then compile (or is this a regression issue
>>>> which I failed to grasp).  Or do I need to update binutils on my
>>>> *host* system itself?  i.e. what I'm really asking is does make world
>>>> bootstrap the right bintuils/gcc etc. and then use THAT to compile the
>>>> rest or does it just perform a host build of everything and plops it
>>>> in DESTDIR?
>>>>
>>>> Hope I make some sense here (still a n00b)....
>>>
>>> One thing we always strive for in FreeBSD is an upgrade path.
>>>
>>> As a general rule, a newer system should be able to run a jail
>>> populated with an earlier system. There are some small exceptions,
>>> for example you may need a new version of netstat, ps and libkvm
>>> in your jail. possibly grab them from the /rescue on the new system
>>> so they are statically linked.
>>> also 8.x systems will require that threaded programs from 6.x be
>>> dynamically
>>> linked so that they can be remapped to use libthr instead of libkse as
>>> libkse is not supported in 8.
>>
>> So you are talking about binary/ABI compatibility yes?  So I would
>> assume what you are saying is I can take a 6.x system, create a
>> filesystem tarball, drop it on a 7.x system and then create a jail out
>> of it.
>>
>>> asside from those I think that just about every thing else should be
>>> fine..
>>> I've run a FreeBSD 1.1 chroot on a freeBSD 7 system
>>> (I had to make 1 very small fix).
>>>
>>> At Ironport we build 4.x binaries on 6.x systems by spinning off
>>> a 4.x chroot as prart of the build process. (they need to link with 4.x
>>> third party binaries) so it's very esay to do.
>>
>> I believe this answers my question but I want to confirm.  I THOUGHT
>> about this but I wanted a more *cleanroom* approach.  That's all.
>
> this is about as cleanroom as you can get..
> you create a tarball of a virgin 6.x distribution including
> virgin stuff you need, and then every time you need to do a
> new 6.x jail, you just drop it in and hook it up.
>
> it doesn't even need to be a jail if you are just building..
> a chroot should be good enough. (with a /dev mounted in it usually)
> (for /dev/null and such).

Alright Julian, sounds reasonable.  I just wanted to confirm on the
list that this is really acceptable and I won't (generally speaking)
run into any ABI issues (clearly on other platforms, this is NOT
always the case!).

Thanks again guys,

-aps