parallel builds revisited

Benjamin Lutz mail at
Tue Apr 10 03:04:47 UTC 2007


Some time ago, after buying a Core 2 Duo system, I've become interested 
in doing something about the inherent single-threadedness of the ports. 
Even though I have a dualcore machine, ports builds only ever use one 
core. I started thinking about various approaches to introduce 
parallelism to ports builds and wrote down my thoughts here:

Well, since then I've tinkered with various approaches. I concentrated 
on using make's -j feature. After adding the flag to the gmake 
invocation in, I quickly noticed that some ports can take 
advantage of the flag and thus build much more quickly (eg, all the KDE 
ports), others are still single-threaded (, but of course there 
are also ports that fail to build ( This means that a 
per-port switch is required.

To make this change as as unintrusive as possible, and to not have to 
patch a large number of files after every portsnap, I've decided to 
build a whitelist of ports which work with (or despite? :) ) parallel 
building. So here's what I use currently:

  1) Put "USE_LOCAL_MK=yes" into /etc/make.conf

  2) Use this as /usr/ports/Mk/ .
     (if you want just my changes in that file, you can run 
     "grep PARALLEL")

  3) Save this to /usr/local/etc/parallel_builds.conf: .
     This is a list of ports as stored in PKGORIGIN, or as
     pkg_info -o reports them.

The code will run make with -jx, where x is the number of cores your 
machine has, as reported by kern.smp.cpus.

So now I would like to invite you test, comment, or simply philosophize 
on these changes.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
Url :

More information about the freebsd-ports mailing list