svn commit: r307680 - in head/tools/tools: . git
Ryan Stone
rstone at FreeBSD.org
Thu Oct 20 18:23:45 UTC 2016
Author: rstone
Date: Thu Oct 20 18:23:44 2016
New Revision: 307680
URL: https://svnweb.freebsd.org/changeset/base/307680
Log:
Add some tools to simplify the use of git
Add some scripts that wraps some FreeBSD Project infrastructure
and simplifies using them with git. The scripts are:
- arcgit, which creates a series of reviews in Differential
- importgit, which applies a series of git commits to svn
Differential Revision: https://reviews.freebsd.org/D2071
Added:
head/tools/tools/git/
head/tools/tools/git/HOWTO (contents, props changed)
head/tools/tools/git/arcgit (contents, props changed)
head/tools/tools/git/importgit (contents, props changed)
Modified:
head/tools/tools/README
Modified: head/tools/tools/README
==============================================================================
--- head/tools/tools/README Thu Oct 20 17:28:52 2016 (r307679)
+++ head/tools/tools/README Thu Oct 20 18:23:44 2016 (r307680)
@@ -29,6 +29,7 @@ gdb_regofs A simple tool that prints out
platforms only.
genericize Turn a kernel config into something that can more easily
be diffed against the appropriate GENERIC.
+git Tools to simplify the use of git by committers.
hcomp Compress header files by removing comments and whitespace.
html-mv Rename HTML generated filenames to human readable filenames.
ifinfo Uses the interface MIB to print out all the information
Added: head/tools/tools/git/HOWTO
==============================================================================
--- /dev/null 00:00:00 1970 (empty, because file is newly added)
+++ head/tools/tools/git/HOWTO Thu Oct 20 18:23:44 2016 (r307680)
@@ -0,0 +1,144 @@
+# $FreeBSD$
+
+This directory contains tools intended to help committers use git when
+interacting with standard FreeBSD project resources like Differential or svn.
+
+I. arcgit
+
+arcgit is a wrapper script around the arc command line tool that simplifies the
+automatic creation of a series of Differential reviews from a series of git
+commits. The intended workflow is:
+
+1. Create a series of commits in git. Each commit will be a separate review, so
+ try to make each commit as self-contained as possible. The series of commits
+ should demonstrate a logical progression towards your end goal. For example,
+ one commit might refactor existing code to expose a new API without changing
+ any current functionality. A subsequent commit could then introduce your new
+ code that uses the new API.
+
+ It usually will not be helpful to present your code in the order in which it
+ was actually written and can actually be harmful. For example, if you
+ introduced a bug early in your development process that you fixed in a
+ subsequent commit, it is a waste of your reviewer's time to have them review
+ old code with known bugs. Instead, it would probably be best to squash the
+ bug fix into the commit that introduced it, so that the bug is never
+ presented to your reviewers in any review.
+
+ The commit headline and commit message will be imported verbatim into
+ Differential, so try to give each commit a meaningful commit message that
+ gives your reviewers the necessary context to understand your change.
+
+2. Create your reviews bu running this command in your git repo:
+ $ arcgit -r C1~..C2 -R reviewer -T testplan
+
+ C1 should be the first commit that you want reviewed, and C2 should be the
+ last commit that you want reviewed. You may add multiple reviewers by
+ specifying the -R option multiple times. You can CC (AKA subscribe) people
+ to a review with the -C option. Note that if you subscribe a mailing list
+ to a review, the mailing list will be emailed for every comment or change
+ made to each review. Please be judicious when subscibing mailing lists to
+ reviews. It may be better to instead send a single email to the appropriate
+ list announcing all of the reviews and giving a short summary of the change
+ as a whole, along with a link to the individual reviews.
+
+3. When you need to make a change and upload it to a review, use the following
+ process. First, check out the branch corresponding to the review (arcgit
+ automatically creates this branch for every review that it creates):
+
+ $ git checkout review_D1234
+
+ Next, make your change and perform whatever testing is necessary. Commit it
+ to your repository with this command:
+
+ $ git commit --fixup HEAD
+
+ You can upload the change to the code review by running this command in your
+ repository while (ensure that you are still on the review_D1234 branch):
+
+ $ arc diff --update D1234 review_D1234_base
+
+ When you run arc, it will pull up your editor and give you a chance to
+ change the message that will be shown in Differential for this upload. It's
+ recommended that you change it from the default "fixup! ...." as that does
+ not give your reviewers an indication of what changes were made. It's not
+ recommended that you give the code review fixes meaningful commit messages
+ directly because we will be using git's autosquash feature in the next step,
+ which depends on the fixup! tag being present.
+
+ Do not merge in other branches, or rebase your review branches at this point.
+ Any changes that are made will show up in your reviews, and that will create
+ extra noise that your reviewers will have to ignore. If a reviewer requests
+ a change that requires your commit to be based off of a later version of
+ head, I would suggest deferring the change from the review and creating a
+ new review with the change once you hit step 5.
+
+4. Once the reviews have been approved, you need to prepare your patch series
+ to be committed. This involves squashing the fixes made in code review
+ back into the original commit that they applied to. This gives you a clean
+ series of commits that are ready to be commited back to svn.
+
+ First, merge each of your review branches back into your main development
+ branch. For example:
+
+ $ git checkout myfeature_dev
+ $ for branch in review_D1234 review_D1235 review_D1236; do \
+ git merge $branch || git mergetool -y || break; done
+
+ Next, do an interactive rebase to squash your code review fixes back into the
+ main commit:
+
+ $ git rebase -i -k review_D1234_base
+
+ review_D1234 should be the name of the *first* review that was created for
+ you in step 2. For every commit, your editor will be pulled up and you will
+ be given one last chance to edit your commit message. Make sure that you fill
+ in the "Reviewed by:" tag indicating who accepted the review. This would
+ be a good point to add other tags like MFC after:, Sponsored by:, etc.
+
+ The rebase will not introduce any actual changes to your code if done
+ properly. You can use this command to double-check that no changes were
+ inadvertently made here:
+
+ $ git diff myfeature_dev@{1}
+
+5. Finally, you should get up to date with the latest changes from head:
+
+ $ git pull --rebase origin master
+
+ It is not recommended that you combine this step with the rebase done in the
+ previous step. The reason for this is that if you perform an interactive
+ rebase that changes the commit that you branch from, it becomes much more
+ difficult to use the reflog to confirm that the interactive rebase did not
+ introduce unwanted changes.
+
+ At this point, you are ready to commit your changes to head. The importgit
+ script can be used to import your commits directly into git.
+
+II. importgit
+
+importgit is a script that can take a series of commits from git and commit them
+to a svn repository. The script uses the git commit messages for the svn commit
+message, which allows importgit to be fully automated. This does mean that once
+you start importgit, it will start commit things to svn without giving any
+further chance to sanity check what it's doing.
+
+importgit only supports importing commits that add or modify files. It does not
+support importing commits that rename or delete files, to ensure that git's
+rename detection heuristics do not introduce an error in the import process.
+importgit also does not support importing merge commits. Only linear history
+can be imported into svn.
+
+importgit must be run from a clean subversion checkout. You should ensure that
+the working tree is up-to-date with "svn up" before running importgit.
+importgit will run svn directly, so make sure that your ssh-agent is running
+and has your ssh key loaded already. Run importgit as follows:
+
+ $ importgit -r D1~..D2 -g /path/to/git/repo
+
+This will import every commit between D1 and D2, including both D1 and D2. The
+invocation is very similar to the invocation given to arcgit but there is an
+important point to note. When you rebased your commits as you followed steps 4
+and 5, the commit hashes of all of your commits changed, including C1 and C2.
+You must go back and find the new commit hashes of your commits to pass to
+importgit. Passing -r C1~..C2 would import your commits as they were *before*
+your code review fixes were applied.
Added: head/tools/tools/git/arcgit
==============================================================================
--- /dev/null 00:00:00 1970 (empty, because file is newly added)
+++ head/tools/tools/git/arcgit Thu Oct 20 18:23:44 2016 (r307680)
@@ -0,0 +1,214 @@
+#!/bin/sh
+#
+# Copyright (c) 2015 Ryan Stone. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+# 1. Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# 2. Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution.
+#
+# THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+# ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
+# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+# OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+# OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+# SUCH DAMAGE.
+#
+# $FreeBSD$
+
+# This script is used to submit a series of git commits to Differential. Each
+# commit is submitted as a separate review. For each review, this script will
+# create a branch called review_DXXXX (e.g. review_D2185 for Differential
+# revision D2185). When you need to make a change to a review, checkout the
+# review_D2185 branch, commit your change with "git commit --fixup HEAD". To\
+# upload the change to Differential, use the command:
+# $ arc diff --update D2185 review_D2185_base
+#
+# When your reviews are complete, merge all of the review_DXXXX branches
+# together, and then do a git rebase -ik to meld the code review fixes into the
+# commit that they fixed. Now you have a clean series of patches to commit to
+# svn.
+
+usage()
+{
+ echo "Usage: arcgit <-c commit | -r commit1~..commit2> [-R reviewer] " >&2
+ echo " [-C subscriber] [-T testplan] [-n]" >&2
+}
+
+error()
+{
+ echo "$@" >&2
+ usage
+ rm -f $phab_before $phab_after $arc_msg
+ exit 1
+}
+
+create_review()
+{
+ local commit phab_id arc_dir
+ unset phab_before phab_after arc_msg
+ commit=$1
+
+ phab_before=`mktemp -t arcoutput`
+ phab_after=`mktemp -t arcoutput`
+ echo "Create review for '`git show $commit -s --oneline`'"
+
+ if [ -n "$dry_run" ]
+ then
+ return
+ fi
+
+ git checkout $commit > /dev/null || error "Could not checkout $commit"
+
+ arc_dir="$(git rev-parse --show-toplevel)/.git/arc"
+ arc_msg="$arc_dir/create-message"
+ mkdir -p $arc_dir
+ git show -s --format='%B' HEAD > $arc_msg
+ echo >> $arc_msg
+ echo "Test Plan:" >> $arc_msg
+ cat $test_plan >> $arc_msg
+ echo >> $arc_msg
+ echo "Reviewers:" >> $arc_msg
+ echo "$reviewers" >> $arc_msg
+ echo >> $arc_msg
+ echo "Subscribers:" >> $arc_msg
+ echo "$cc_list" >> $arc_msg
+ echo >> $arc_msg
+
+ arc list > $phab_before
+ yes | env EDITOR=true arc diff --create --allow-untracked HEAD~
+ arc list > $phab_after
+
+ headline="$(git show $commit -s --format=%s)"
+ phab_id=`comm -13 "$phab_before" "$phab_after" | fgrep "$headline" \
+ | egrep -o 'D[0-9]+:' | tr -d ':'`
+
+ if [ -z "$phab_id" ]
+ then
+ error "Could not get review ID"
+ fi
+
+ git branch review_${phab_id}_base HEAD~
+
+ git checkout -b review_$phab_id
+ cat - <<EOF | git commit --allow-empty -F -
+squash! $headline
+
+Differential Revision: https://reviews.freebsd.org/$phab_id
+Reviewed by:
+EOF
+}
+
+unset range test_plan reviewers cc_list dry_run
+
+while getopts ":c:C:nr:R:T:" o
+do
+ case "$o" in
+ c)
+ range="${OPTARG}~..${OPTARG}"
+ ;;
+ C)
+ if [ -z "$cc_list" ]
+ then
+ cc_list="$OPTARG"
+ else
+ cc_list="$cc_list, $OPTARG"
+ fi
+ ;;
+ n)
+ dry_run=1
+ ;;
+ r)
+ range=$OPTARG
+ ;;
+ R)
+ if [ -z "$reviewers" ]
+ then
+ reviewers="$OPTARG"
+ else
+ reviewers="$reviewers, $OPTARG"
+ fi
+ ;;
+ T)
+ test_plan=$OPTARG
+ ;;
+ *)
+ error "Unrecognized argument '-$OPTARG'"
+ esac
+done
+
+shift $((OPTIND - 1))
+OPTIND=1
+
+if [ -n "$1" ]
+then
+ error "Unrecognized argument $1"
+fi
+
+if [ -z "$range" ]
+then
+ error "-c or -r argument is mandatory"
+fi
+
+if [ -n "$test_plan" -a ! -r "$test_plan" ]
+then
+ error "$test_plan is not readable"
+fi
+
+if ! type git > /dev/null 2> /dev/null
+then
+ error "Install devel/git first"
+fi
+
+if ! type arc > /dev/null 2> /dev/null
+then
+ error "Install devel/arcanist first"
+fi
+
+git update-index -q --refresh
+if ! git diff-index --quiet --cached HEAD
+then
+ error "index is unclean"
+fi
+
+if ! git diff-files --quiet
+then
+ error "Working directory is unclean"
+fi
+
+if git ls-files --other --error-unmatch . > /dev/null 2> /dev/null
+then
+ error "Working directory contains untracked files"
+fi
+
+# We have to do a git checkout in order to run arc, so save the original branch
+# so that we can check it out again once we're done.
+if ! orig_branch=$(git symbolic-ref --short -q HEAD)
+then
+ orig_branch=$(git show -s --pretty='%H' HEAD)
+fi
+
+git log --format=%H $range | tail -r | while read -r commit
+do
+ create_review $commit < /dev/null
+done
+
+# Note that due to the use of the pipeline above, the body of the while loop
+# above runs in a subshell. If it exits with an error, execution resumes
+# here rather than exiting the script, so we have to cache the right exit code
+# and return it when we're done cleaning up.
+code=$?
+
+git checkout $orig_branch
+
+exit $code
+
Added: head/tools/tools/git/importgit
==============================================================================
--- /dev/null 00:00:00 1970 (empty, because file is newly added)
+++ head/tools/tools/git/importgit Thu Oct 20 18:23:44 2016 (r307680)
@@ -0,0 +1,182 @@
+#!/bin/sh
+#
+# Copyright (c) 2015 Ryan Stone. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+# 1. Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# 2. Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution.
+#
+# THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+# ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
+# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+# OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+# OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+# SUCH DAMAGE.
+#
+# $FreeBSD$
+
+usage()
+{
+ echo "Usage: importgit <-c commit | -r c1..c2> -g /path/to/git/repo [-n]" >&2
+}
+
+error()
+{
+ local print_usage
+
+ if [ "$1" = "-u" ]
+ then
+ shift
+ print_usage=1
+ else
+ print_usage=
+ fi
+
+ echo "$@" >&2
+ if [ -n "$print_usage" ]
+ then
+ usage
+ fi
+ exit 1
+}
+
+unset git range commit dry_run
+
+while getopts ":c:g:nr:" o
+do
+ case "$o" in
+ c)
+ range="${OPTARG}~..${OPTARG}"
+ ;;
+ g)
+ git_repo=$OPTARG
+ ;;
+ n)
+ dry_run=1
+ ;;
+ r)
+ range=$OPTARG
+ ;;
+ *)
+ error -u "Unrecognized argument '-$OPTARG'"
+ esac
+done
+
+shift $((OPTIND - 1))
+OPTIND=1
+
+if [ -n "$1" ]
+then
+ error -u "Unrecognized argument $1"
+fi
+
+if [ -z "$range" ]
+then
+ error -u "-c or -r argument is mandatory"
+fi
+
+if ! echo "$range" | egrep -qs '^[^.]+\.\.[^.]*$'
+then
+ error -u "$range is not a range of commits. Did you mean '-c $range'?"
+fi
+
+if [ -z "$git_repo" ]
+then
+ error -u "-g <repo> argument is mandatory"
+fi
+
+git="$git_repo/.git"
+
+if [ ! -d "$git" ]
+then
+ error "$git_repo does not seem to be a git repo"
+fi
+
+if ! type git > /dev/null 2> /dev/null
+then
+ error "Install devel/git first"
+fi
+
+if ! type svn > /dev/null 2> /dev/null
+then
+ error "Install devel/subversion first"
+fi
+
+if [ -n "$(svn status)" ]
+then
+ error "Working tree is not clean"
+fi
+
+if ! svn --non-interactive ls > /dev/null
+then
+ error "Could not communicate with svn server. Is your ssh key loaded?"
+fi
+
+git --git-dir=$git log --format=%H $range | tail -r | while read -r commit
+do
+ echo "Applying `git --git-dir=$git show -s --oneline $commit`"
+
+ if [ -n "$(git --git-dir=$git show --diff-filter=CDRTUXB $commit)" ]
+ then
+ error "Commit performed unsupported change (e.g. delete/rename)"
+ fi
+
+ if [ "$(git --git-dir=$git show -s --format=%P $commit | wc -w)" -ne 1 ]
+ then
+ error "Cannot import merge commits"
+ fi
+
+ git --git-dir=$git diff --diff-filter=A --name-only \
+ ${commit}~..$commit | while read -r newfile
+ do
+ if [ -f "$newfile" ]
+ then
+ error "New file $newfile already exists in tree"
+ fi
+ done
+
+ # The previous while loop ran in a subshell, so we have to check if it
+ # exited with an error and bail out if so.
+ ret=$?
+ if [ "$ret" -ne 0 ]
+ then
+ exit $ret
+ fi
+
+ if [ -n "$dry_run" ]
+ then
+ continue
+ fi
+
+ git --git-dir=$git show $commit | patch -p 1 -s || \
+ error "Failed to apply patch"
+
+ git --git-dir=$git diff --diff-filter=A --name-only \
+ ${commit}~..$commit | while read -r newfile
+ do
+ svn add --parents --depth=infinity $newfile || \
+ error "Failed to add new file"
+ done
+
+ # The previous while loop ran in a subshell, so we have to check if it
+ # exited with an error and bail out if so.
+ ret=$?
+ if [ "$ret" -ne 0 ]
+ then
+ exit $ret
+ fi
+
+ git --git-dir=$git show -s --format='%B' $commit | svn commit -F - || \
+ error "Failed to commit"
+done
+
More information about the svn-src-all
mailing list