Parallel Builds

[Ulrich Spoerlein]

Doug Barton wrote:
> > o Leave the ports framework as it is, and implement support for
> >   parallel building in add-on tool, eg., portupgrade. The tool would
> >   support automatic parallelism ("portupgrade -a" would automatically
> >   build ports in parallel where possible), or having several
> >   user-created instances running at the same time. I'll call this
> >   "tool-based macro-parallelism".
> 
> I specifically designed portmaster to be able to do this. Separate instances do not share 
> anything in common, other than the ports tree and package database. Assuming that you were 
> updating two (or more) ports that you were positive did not share any dependencies in common, 
> you could run as many portmaster instances as you wanted to, and not have any problems.
> 
> Where this gets really sticky is in (for example) the -a case. If you run 'portmaster -l' on 
> a typical system, you'll see that there are some root ports (no dependencies, not depended 
> on), some leaf ports (not depended on), but most of the installed ports are in the "middle," 
> where they are depended on by something else, and most of those have dependencies of their 
> own. It would be a SMOP to have portmaster track the ports that are "safe" to upgrade in 
> parallel. Then of course you have to keep track of your child processes, create a queue for 
> what ports to run next, etc. etc. It's all doable of course, but I think the return on 
> investment for this project would be very small. Patches are always welcome however. :)
> 
> >   Disadvantages:
> >   - Moderately difficult to implement.
> 
> *cough*

I have a small script in production, which grabs all dependant ports to
compile and does a serial package build, where all required packages are
built/installed prior to the next package being built.

With some minor additions, I *think* I could make those parallel package
builds work without too much fuss.

Package upgrade a something different though, but the whole ports
framework is flawed in that area anyway ('make package' should come
*before* 'make install').

Don't hold your breath though, ENOTIME as always ...

Ulrich Spoerlein
-- 
A: Yes.
>Q: Are you sure?
> >A: Because it reverses the logical flow of conversation.
> >>Q: Why is top posting frowned upon?