Request for Features: Ports Re-engineering

[Mark Linimon]

On Mon, Dec 17, 2007 at 12:59:54PM -0300, Alejandro Pulver wrote:
> if not already done, AFAIK tinderbox has an option to run parallel
> builds, but don't know about pointyhat) using multiple processors.

pointyhat is actually the dispatch machine.  It does not actually
build packages.

The current machine total:

amd64:    3 (2.2GHz)
i386:     40 (400MHz - 1.8GHz, depending on machine)
sparc64:  7 (440MHz)

Several of these systems (in particular the amd64s) have multiple CPUs in them.

In any case, each machine is given several packages to work on simultaneously,
depending on its capacity, each in its own fully clean jail.  Prerequisites
are loaded via the previously built packages; if they are not available,
they are first built, then cached.

The way the current code works is documented in the following article:

http://www.freebsd.org/doc/en_US.ISO8859-1/articles/portbuild/index.html

Although the documentation is terse, it is believed to be generally
correct.

In general, we use -incremental for the builds, unless there has been
a large change in the src tree, or it's release.  -incremental will only
build packages for ports whose metadata (as determined by INDEX) has
changed since the last run.  This holds the build time for each package
set down to a few days (amd64/i386) or a couple of weeks (sparc64),
instead of around a week, or a couple of months, respectively.

Remember that we have builds for 5.x, 6.x, 7.x, and 8.x all running
at various times (some things can be overlapped without adding too much
delay).  The machines are almost never idle.

I strongly suggest that before designing a new system, some careful
attention be paid to how extensive and optimized the current system already
is.  I don't think the people following this thread are going to get the
right perspective on the scope of this work, otherwise.

mcl