parallel builds revisited

[Benjamin Lutz]

Hello,

Some time ago, after buying a Core 2 Duo system, I've become interested 
in doing something about the inherent single-threadedness of the ports. 
Even though I have a dualcore machine, ports builds only ever use one 
core. I started thinking about various approaches to introduce 
parallelism to ports builds and wrote down my thoughts here:
http://marc.info/?l=freebsd-ports&m=116124997126657&w=2

Well, since then I've tinkered with various approaches. I concentrated 
on using make's -j feature. After adding the flag to the gmake 
invocation in bsd.port.mk, I quickly noticed that some ports can take 
advantage of the flag and thus build much more quickly (eg, all the KDE 
ports), others are still single-threaded (X.org), but of course there 
are also ports that fail to build (Openoffice.org). This means that a 
per-port switch is required.

To make this change as as unintrusive as possible, and to not have to 
patch a large number of files after every portsnap, I've decided to 
build a whitelist of ports which work with (or despite? :) ) parallel 
building. So here's what I use currently:

  1) Put "USE_LOCAL_MK=yes" into /etc/make.conf

  2) Use this as /usr/ports/Mk/bsd.local.mk:
     http://www.maxlor.com/temp/bsd.local.mk .
     (if you want just my changes in that file, you can run 
     "grep PARALLEL bsd.local.mk")

  3) Save this to /usr/local/etc/parallel_builds.conf:
     http://www.maxlor.com/temp/parallel_builds.conf .
     This is a list of ports as stored in PKGORIGIN, or as
     pkg_info -o reports them.

The code will run make with -jx, where x is the number of cores your 
machine has, as reported by kern.smp.cpus.

So now I would like to invite you test, comment, or simply philosophize 
on these changes.

Cheers
Benjamin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-ports/attachments/20070410/38483212/attachment.pgp