parallel builds revisited

[youshi10 at u.washington.edu]

On Thu, 12 Apr 2007, [LoN]Kamikaze wrote:

> Benjamin Lutz wrote:
>> On Thursday 12 April 2007 11:06, Garrett Cooper wrote:
>>> I dunno how you want to approach this, but gmake does recommend 2
>>> jobs be run in parallel for HTT enabled chips, and 3 or 4 jobs for a
>>> dual core machines.
>>> -Garrett
>>
>> So far the approach is one job per CPU. I'll do some benchmarks lateron
>> to determine wether it really helps to run more jobs. For the KDE
>> ports, my gut feeling is that the improvement would be negligible. I'll
>> have to evaluate non-C++ ports like gnome-*, where the compilation time
>> per file is shorter.
>
> I find the benefit especially big with Gecko based ports like Firefox or
> Thunderbird.
>
>> Of course, to make proper use of distcc, at least #cores + 1 jobs are
>> required. I'll keep that in mind.
>
> The recommendation of the Distcc developers is 2 jobs per core, as long as
> the machines contain sufficient memory.
>
> My personal experience says that it's rarely possible to divide a Makefile
> into more than 6 threads.

Perhaps, but that's most likely just a limitation behind gmake. With a bit of a rewrite by the devs it could better utilize lookahead and dependency mapping I would think than it currently does.

Needless to say this could be an easily squashed issue for making builds go faster, particularly with the large multicore / multiprocessor machines present nowadays.

-Garrett