HEADS UP multi processor compilations for everyone

[Yoshihiro Ota]

On Wed, 25 Mar 2009 12:57:51 -0400
Coleman Kane <cokane at FreeBSD.org> wrote:

> On Wed, 2009-03-25 at 19:30 +0300, Dmitry Marakasov wrote:
> > * Pav Lucistnik (pav at FreeBSD.org) wrote:
> > 
> > > > > This would break very fast -- it's passing -j3 to port Makefile instead
> > > > > of vendor Makefile.
> > > > 
> > > > This has worked fine for me for countless years, except where the
> > > > vendor's Makefiles were not parallel-safe. This has been my trick to get
> > > > larger things (like mysql or xorg-server) to make in parallel. It *did*
> > > > work. If this has changed, then it definitely warrants mention in
> > > > UPDATING.
> > > 
> > > Then it must have worked all these years by pure chance :)
> > 
> > Be the way, could anyone clarify how this works? My idea was that
> > passing -j to port Makefile does nothing, as make/gmake on vendor's
> > Makefile is ran without any -j flags -> you get usual singlethreaded
> > build. However, I have a broken port, which uses gmake and something
> > like that:
> > 
> > sometarget:
> >   (cd xxx; make)
> > 
> > and that fails with -j (make: illegal option -- -). So is there
> > some magic with recursive make calls and -j?
> > 
> 
> When processing a Makefile, make's that support concurrent operation
> look for targets that will execute the $(MAKE) program. Whenever a
> compliant make is run (make or gmake), if it detects $(MAKE) in a rule
> then it will automatically expand that rule into a child process that
> has some sort of magical connection to the parent process. The
> connection allows the different make processes to share the same pool of
> "process count" resources amongst each other.
> 
> I am not sure what the implementation is, but this communication
> mechanism allows child "make" processes called with "$(MAKE) ..." from
> inside a Makefile to globally only use N children (from -j N), and
> otherwise block until more "free jobs" are available amongst their
> shared job pool.
> 
> I hope that's clear... Probably more so than the explanation given on
> GNU make's manual.
> 
> -- 
> Coleman Kane
> 

Indeed, that is why (cd xxx; make) fails where gmake is running this command.
The way GNU make and BSD make passes -j related options are different.  They are
not compatible.  If I remember correctly, if GNU make calls BSD make (which
is an error of writing Makefiles in most of vendor codes), the differences are
significant and fails.  Usually, changing "make" to "$(MAKE)" fixes the problems.

Why did it build when "-j" was not supplied.  That is because makefile rule was
not GNU makefile specific such that BSD make could also executed it without problems.
With "-j" switch presented, these two "make" commands became incompatible each
other.

I hope this helps if you guys haven't found this facts; I hope I had replied earlier.

Regards,
Hiro