cvs commit: ports/Tools/portbuild/scripts dopackages

Kris Kennaway kris at
Sat Jul 26 14:49:26 UTC 2008

kris        2008-07-26 14:49:26 UTC

  FreeBSD ports repository

  Modified files:
    Tools/portbuild/scripts dopackages 
  * Cleanup
  * Catch up to build ID directory changes
  * Improve usage()
  * Fix a variety of small bugs
  * Remove support for -ftp builds: we have not supported direct
    uploading for many years due to the desire to manually inspect build
    output for quality
  * All data associated to a build is now localized in its own directory
    named according to a build ID:
    /var/portbuild/${arch}/${branch}/builds/${buildid}, where ${buildid}
    is the creation time.  These are actually ZFS filesystems.
  * Tasks such as cloning a new build, updating a ZFS snapshot, and
    cleaning up a build are exported to the "build" script, which can be
    used independently.
  * Creating a new build is done by ZFS cloning and takes a couple of
    seconds since it is copy-on-write (i.e. no data needs to be copied).
  * Ports and source trees are also cloned from pre-updated ZFS images
    (updated regularly from the "updatesnap" cron job).  In most cases
    we do not care if we are building a ports tree that is an hour or so
    old since it will become outdated almost immediately anyway, so no
    matter what we do there will be times when a port has been fixed by
    the time the build error is generated by a client.
  * In case an up-to-the-second tree is desired, the -portscvs and
    -srccvs switches update the existing ports tree via CVS.
  * -noports and -nosrc can be used to prevent any automatic changes to
     the ports tree.  This is useful for dealing with local
     modifications (e.g. for -exp builds), since the default when
     creating a new build is to replace the previous trees with fresh,
     pristine trees.  If you forget to use this then any local changes
     that are not also present in other trees will be lost.
  * By default we keep two builds for each arch/branch pair.  These
    build IDs also may be referred to via "latest" and "previous"
    symlinks.  When creating a new build, the old "previous" build is
    destroyed by default, unless it was originally created using the
    -keep switch.  This prevents the build from being destroyed
  * By default when a build finishes all of the clients are completely
    cleaned up (i.e. all build data such as ports trees, tarballs,
    client chroots, etc are deleted).  This is needed to save space on
    the clients.  If you expect to *immediately* perform further builds
    after this one completes, the -nocleanup switch prevents this step.
    Otherwise they will just be set up again if further builds are
  * Try to parallelize build pre-processing as much as possible, by
    running jobs in the background wherever possible.  In several places
    we operate on the same parts of the filesystem from multiple jobs,
    so we can make good use of caching to improve performance
  * Clients no longer need to be set up explicitly at the start of the
    build, they will be set up on-demand when the first job is
    dispatched to them.  This allows fast clients or those that already
    have been set up to begin building ports as soon as possible, while
    slow clients are set up in the background.  It also improves
    robustness of client recovery, e.g. if the client was offline at the
    time of build startup but later brought back online.
  * Optimize copying back in the previous set of restricted packages by
    hardlinking instead of copying.
  TODO: The record of failed ports is arch/branch-global still.  This is
  the only thing preventing us from running concurrent builds of the
  same arch/branch (e.g. while one is stuck building openoffice, the
  next build can start to keep the cluster busy).  The difficulty is
  that one build from a later ports tree may signal that a build was
  successful, then a phase 2 build from an earlier ports tree may
  indicate that it was broken.  The solution is probably to migrate this
  to a real database instead of a flat file, and query it for the set of
  broken ports as of a certain ports tree date.
  Revision  Changes    Path
  1.48      +586 -501  ports/Tools/portbuild/scripts/dopackages

More information about the cvs-all mailing list