Parallel builds, PKG_DBDIR locks

[David Forsythe]

Hey all,

Last year for my Summer of Code project, I worked on adding parallel build
support to ports.  While it worked, some of it didn't work right, and I
wasn't really happy with the final result.  I finally got some free time
lately and decided to clean it up a little bit, and since it's at a point
where I can use it without fearing cataclysmic failure, I figured I'd share
with anyone else who might be interested.

What bsd.locked.mk allows you to do first and foremost is build more than
one port at a time without fearing for your PKG_DBDIR or overwrites when
ports with common dependencies build as long as you have the USE_LOCKS flag
set.  It creates a lock in a building ports directory that, if kept
consistent, will prevent the port from being built more than once at a time.
 When it comes time for a port to register itself (fake-pkg), PKG_DBDIR
itself is used as a lock to prevent inadvertent clobbering of the database.
 I also added a locking for fetches, so that if ports share distfiles (qt4
stuff, for instance), your builds won't overwrite the files or outright
fail.  With P_DEPENDS and P_FETCH, you have the option of parallel
dependency handling and distfile fetching.  P_DEPENDS=4, for instance, will
allow a port to build up to 4 dependencies at a time.  P_FETCH=3 will allow
up to 3 distfiles of a port to be fetched at a time.

I've generated a patch for anyone who wants to test this out.  If the patch
fails to apply cleanly, just grab the makefile and add the conditional for
USE_LOCKS right before the master-sites-ALL target.  If you don't have
USE_LOCKS set, everything will work as if bsd.locked.mk isn't even there.  I
had a patch from last summer that made pkg_install and pkg_deinstall honor
the locks as well, but I'm not sure where I put it, and it's probably no
good at this point anyway.  It was pretty simple, regardless, and I could
redo it if anyone actually takes an interest in this.

Run make config-recursive or use BATCH if you use this or else you might be
some nasty suprises.

patch:  http://bsdtips.utcorp.net/~dforsyth/bsd.locked.mk.diff (apply it in
your ports makefile directory)

makefile:  http://bsdtips.utcorp.net/~dforsyth/bsd.locked.mk

Thanks,

Dave
-- 
David Forsythe