make -j$n buildworld : use of -j investigated

[Garance A Drosihn]

At 12:51 PM +0900 11/23/04, Rob wrote:
>Laurence Sanford wrote:
>>Rob wrote:
>>
>>>----------------------------------------
>>>
>>>With these simple tests, I come to the conclusion that
>>>"make -j$n buildworld" is best with n = number of CPUs.
>>>Does that make sense?
>>>
>>>Rob.
>>
>>This is what I've been telling people and using myself for
>>years. However, I've been shot down on this several times,
>>so I just leave everyone alone and let them do their own
>>thing. You and I will be getting it done a little faster
>>though.
>
>Not really faster, but higher values do not make a difference,
>well, as long as the extra processes do not force the use of
>swap. Intensified swapping because of a high -j value slows
>down the build considerably.
>
>I don't understand why this is reason for debate. My test has
>obvious results on various of my PCs, and was very quickly done:
>I wrote a script with a loop that built the world again and
>again, doing a 'touch' to a file immediately before and after
>the build. Got all my data within a day or so.

There are many things will effect what different people see for
buildworld times on their own hardware.  You also have to make
some effort to make sure you're doing *exactly* the same thing
on each test build, as your results may be skewed due to various
caching that can go on.

You're not the only one who has benchmarked this for yourself, and
my guess is that everyone who does a serious benchmark will simply
find out the right answer for *their* hardware.  I have had dual-
processor machines where buildworlds kept getting faster up to -j9,
but even -j9 was only about 2% faster than -j4, so I stuck with -j4.
In my case, I tried with -j values from 1 to 15.

On my single processor i386 systems, I generally find that -j2 is
faster than -j1.  Anything more than -j2 and performance seems to
get worse.  On my single processor sparc64 system, the disks are so
slow that there's no point doing -j at all.

So my own rule of thumb is -j of 2*number-of-CPU's (but then most
of my machines have plenty of memory), except for the Ultra-10.

Also, did you do your benchmarking before or after the recent
fixes to -j processing?  I haven't redone them after that change,
and I think it will be interesting to see what effect that has.

-- 
Garance Alistair Drosehn            =   gad at gilead.netel.rpi.edu
Senior Systems Programmer           or  gad at freebsd.org
Rensselaer Polytechnic Institute    or  drosih at rpi.edu