speed up port compiling using RAM (tmpfs) ???

[Peter Jeremy]

On Fri, 2006-Jan-20 14:47:00 -0500, Kris Kennaway wrote:
>On Fri, Jan 20, 2006 at 11:49:29AM -0500, Gary Thorpe wrote:
>
>> >-j is not safe to use with port builds since many ported software
>> >contain race conditions in the build.
>> >
>> >Kris
>> 
>> This effectively means that you cannot take advantage of SMP to compile 
>> FreeBSD's ports collection. That sounds like a big 
>> limitation...especially for people trying to speed up bulk builds.
>
>Yeah, but what do you propose to do about it?  We have 14000 ports
>that need to be inspected for build race conditions and fixed before
>you can turn on -j by default.

There are some areas of the FreeBSD port building infrastructure that
could be enhanced to make it possible to run multiple top-level makes
in parallel, even if individual ports can't be build with '-j'.

IMHO, the biggest problem (as des pointed out) is that there's nothing
to prevent two makes attempting to build the same port (this can
easily happen when both ports A and B depend on port C).  One possible
solution would be to create another status file in the work directory
and hold a file lock (flock/lockf) on it whilst a make is in progress.
Any parallel attempt to make that port would block.

If the above was implemented, an enhancement would then be to process
the port's dependency list in parallel, rather than serially.  This
would allow a make that was blocked on one dependency to continue with
a different dependency.

-- 
Peter Jeremy