Making a local branch of the ports tree

[Vivek Khera]

On Apr 25, 2007, at 1:22 PM, Bill Moran wrote:

> My thought is to make /usr/ports/private (or similar) and teach cvsup
> not to blow it away.  Then I just need to make sure that portupgrade
> and other tools see it.
>
> Does anyone have a HOWTO or list of steps to get this going?  I know
> this has been discussed before but I can't find any reference to it  
> now.

I use /usr/port/local and name all the ports "kci-XXX" for whatever  
port we have.  mostly these are pseudo ports which pull in all the  
dependencies for our various server needs.  (I'll probably post this  
to my website sometime...)

Step 1: mkdir /usr/ports/local
Step 2: create the following /usr/ports/Makefile.local

-- cut here --
# local Makefile additions for top-level ports to add our local  
category.

SUBDIR += local
-- cut here --

So now the ports infrastructure knows about that directory.

Step 3: Inside /usr/ports/local, create a Makefile something like this:

--cut here--
COMMENT = KCI local ports

SUBDIR += kci-base
SUBDIR += kci-monitor
SUBDIR += kci-searchserver

.include <bsd.port.subdir.mk>
--cut here--

Where every subdir of /usr/ports/local is listed here so as to inform  
the ports infrastructure of your local ports.  You must list every  
port you want to be visible in your INDEX file which we will update  
below.

Here is my example /usr/ports/local/kci-base/Makefile which installs  
the bare minimum essentials we need on every server:

--cut here--
PORTNAME=       kci-base
PORTVERSION=    25
PORTREVISION=   0
CATEGORIES=     local
VALID_CATEGORIES=local
MASTER_SITES=   # empty
DISTFILES=      # none
EXTRACT_ONLY=   # empty

MAINTAINER=     khera at kcilink.com
COMMENT=        Base applications needed on all KCI servers

RUN_DEPENDS=    bash:${PORTSDIR}/shells/bash \
                 rsync:${PORTSDIR}/net/rsync \
                 jove:${PORTSDIR}/editors/jove \
                 nano:${PORTSDIR}/editors/nano \
                 gmake:${PORTSDIR}/devel/gmake \
                 portupgrade:${PORTSDIR}/ports-mgmt/portupgrade \
                 portaudit:${PORTSDIR}/ports-mgmt/portaudit

# Don't let stupid programs ask for configuration.

#BATCH=         YES
NO_WRKSUBDIR=   YES
NO_BUILD=       YES

.if make(package)
DEPENDS_TARGET="package"
.endif # .if make(package)

# do nothing here, but can't use NO_INSTALL since it won't make the  
depends
do-install:


.include <bsd.port.mk>
--cut here--


The KEY thing is to set the "VALID_CATEGORIES" to the name of the  
local base ports category, in my case "local".  Other than that, it  
can be just like any other port normally part of the ports tree, and  
needs the necessary support files every port needs.  I haven't been  
able to work out how to do this globally in the /usr/ports/ 
Makefile.local.  If you do, please share.

Now, the only step left is to get your local ports into the INDEX.   
Here is where we use some trickery...  I observed that the make_index  
script from ports uses "make describe" output and cross-references  
stuff to generate INDEX.  However, the INDEX file itself is nearly  
the same format... so if we feed "make describe" from our local  
subtree and concatenate it with the existing INDEX and ignore all  
warnings about make_index not finding port names (since they were  
already substituted in the INDEX file...) we generate the output we  
want.  The following script, localportindex, does this:

--cut here--
#!/bin/sh

# hack to append our local ports to the /usr/ports/INDEX-6 file to  
let the
# portupgrade and portinstall utilities see them.

# Run this after normal update of INDEX file, such as portsnap or  
fetchindex

# $Id: localportindex 480 2006-06-16 15:14:38Z khera $

# for FreeBSD 6.x
PORTSBASE=/usr/ports
PORTSDIRNAME=`cd $PORTSBASE ; /bin/pwd` # for fixingup NFS/symlink  
pathnames
INDEX=${PORTSBASE}/INDEX-6
LOCAL=${PORTSBASE}/local

tmpfile=`/usr/bin/mktemp -t index` || exit 1

# make_index script complains a ton about using the INDEX file rather  
than the
# raw "describe" output so we just toss the errors.  this is probably  
a bad
# thing to do, but nobody will die because of it...
(cd $LOCAL; make describe | sed -e "s@$PORTSDIRNAME@$PORTSBASE at g";  
cat ${INDEX}) | \
perl ${PORTSBASE}/Tools/make_index 2> /dev/null | grep ^kci > $tmpfile

echo "Local ports appended to $INDEX file:"
cat $tmpfile

cat $tmpfile >> ${INDEX}
rm -f $tmpfile
--cut here--


Note that I use the fact all my ports start with "kci-" to pull them  
from the newly generated INDEX file and append that data to the  
actual INDEX file.  This step takes just seconds as opposed to tens  
of minutes (or longer) to regenerate the INDEX from scratch.

All of the above works 100% well with portsnap and cvsup.  I don't  
recall having to teach either how to ignore the local subdir.  If you  
use cvsup, be sure to use some means to have a virgin INDEX file  
before re-running the above index script, like using "make fetchindex".

Here is my "srcupdate" script which updates the source + ports on my  
primary server.  We manage our local ports via subversion, so I just  
ensure that the one in the live ports tree is up-to-date with any  
commits made by any developer.

--cut here--
#!/bin/sh
PATH=/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/local/sbin
cd /usr/src
make update | tee /tmp/update
(cd /usr/ports/local; svn update)
portsnap fetch
portsnap update
portsnap -I update
localportindex
portsdb --update
--cut here--


With all of this in place, portupgrade and friends can manage my  
local ports too.  There is no distinction of what's part of ports and  
what's truly local.