svn commit: r253557 - user/nwhitehorn/condorports

[Nathan Whitehorn]

Author: nwhitehorn
Date: Mon Jul 22 23:08:36 2013
New Revision: 253557
URL: http://svnweb.freebsd.org/changeset/base/253557

Log:
  Remove comment and add documentation.

Added:
  user/nwhitehorn/condorports/README
Modified:
  user/nwhitehorn/condorports/buildportsdag.sh

Added: user/nwhitehorn/condorports/README
==============================================================================
--- /dev/null	00:00:00 1970	(empty, because file is newly added)
+++ user/nwhitehorn/condorports/README	Mon Jul 22 23:08:36 2013	(r253557)
@@ -0,0 +1,90 @@
+condorports
+-----------
+
+This is a version of portbuild designed to exploit the features of the HTCondor
+batch scheduler instead of a homegrown scheduler, in particular the abilities
+of the Condor DAG manager.
+
+Features:
+- Heterogeneous clusters: support builds for multiple architectures using a
+  mixed architecture cluster
+- No shared file system required: distfiles and packages are transferred via
+  condor file transfer to the build node, requiring only static read-only copies
+  of the base system and the ports tree[s] on build nodes
+- Flocking support: if you have multiple Condor clusters, jobs submitted on one
+  can transparently "flock" to another, letting the package builds exploit the
+  resources of geographically distributed computing clusters
+- Many builds for many architectures, OS releases, and ports trees can
+  continue in parallel
+- Uses standard HPC software for scheduling with high performance, reliability,
+  and security with only a light layer on top (155 lines of shell scripts)
+
+Submit node prerequisites:
+--------------------------
+- sysutils/condor, configured to allow job submission
+- Copies of the relevant ports trees, at the same paths as they are on the
+  build nodes
+
+Build node prerequisites:
+-------------------------
+- sysutils/condor, configured using condor_config.local in this directory
+  (adapted for the system in question)
+- security/sudo, with sudoers file in this directory
+- A lot of local disk space in /scratch
+- A copy of all relavent worlds in /releases/<releasenames>
+- The prepbuildjail and reapbuildjail scripts installed in /pkgscripts
+
+Using:
+------
+
+The basic procedure is:
+cd condorports
+./buildportsdag.sh
+condor_submit_dag -maxpre 10 ports.dag
+<Eat one to several meals depending on cluster size>
+
+This will configure a Condor DAG for the ports tree in /usr/ports (the default,
+see the options section below), the current system's architecture
+(again see options), and release (again), placing the packages in the current
+directory (once more). Submitting the DAG to the scheduler will begin the build.
+
+NOTE: The -maxpre option is *very* important, as this controls the number of
+simultaneous make fetch operations to run on the submit node. If you don't
+set it, you will get thousands of fetches at once.
+
+As the build runs, log files will appear in logs and built packages in a
+directory by default in condorports. The build queue can be inspected using
+condor_q.
+
+Options to buildportsdag.sh are specified using environment variables:
+- PORTSDIR: Name of ports directory to build in. This *MUST* occur on the same
+  path on both the build and submit nodes. This directory may be read only.
+  Default: /usr/ports
+- PKGSDIR: Name of directory on submit node in which to place built packages.
+  Need not exist on the build nodes and can be on a local disk.
+  Default: $PORTSDIR/packages
+- DISTHORDE: Name of directory on submit node in which to store and find
+  distfiles. Can be shared across ports tree. Need not exist on the build nodes
+  and can be on a local disk. This directory must be writeable by the user
+  running the job submission.
+  Default: $PORTSDIR/distfiles
+- ARCH: Value of uname -p for which to build
+  Default: Value of uname -p on submit node
+- RELEASE: Name of FreeBSD release to use. This directory must exist on the
+  build nodes (it need not be accessible from the submit node) in
+  /releases/$RELEASE
+  Default: $ARCH/`uname -r`
+- STAGEDIR: Name of directory in which to stage files for transfer for this
+  particular build. Must be unique to this particular package build run and
+  will be deleted at completion.
+  Default: $(pwd)/stage-$ARCH/$PKGSDIR
+
+TODO:
+----
+Not all features are fully implemented yet. In particular:
+- Need to configure networking in the build jails
+- Need to advertise release and ports directory availability in the machine
+  classad (via STARTD_CRON) so that jobs are scheduled only on hosts that
+  have acquired the appropriate resources. This is especially important in
+  the flocking case.
+

Modified: user/nwhitehorn/condorports/buildportsdag.sh
==============================================================================
--- user/nwhitehorn/condorports/buildportsdag.sh	Mon Jul 22 22:16:47 2013	(r253556)
+++ user/nwhitehorn/condorports/buildportsdag.sh	Mon Jul 22 23:08:36 2013	(r253557)
@@ -6,8 +6,6 @@
 : ${RELEASE=$ARCH/`uname -r`}
 : ${STAGEDIR=$(pwd)/stage-$ARCH/$PKGSDIR}
 
-#ports="/usr/ports/games/sl /usr/ports/ports-mgmt/pkg /usr/ports/japanese/nkf"
-
 mkdir -p $STAGEDIR
 
 dagjobs=$(pwd)/ports.dagjobs