Monitoring commits on all branches

Thu Nov 19 21:29:28 UTC 2020

Hi!

On Thursday, November 19, 2020, Dan Langille <dan at langille.org> wrote:

> > On Nov 19, 2020, at 11:16 AM, Marc Branchaud <marcnarc at gmail.com> wrote:
> >
> > On 2020-11-18 8:49 p.m., Dan Langille wrote:
> >> How can a repo be monitored for commits on all branches?
> >> I know how to ask a given branch: do you have any commits after
> foo_hash?
> >> How do I:
> >> * get a list of all commits since foo_hash
> >
> > A quick a note about Warner's reply:
> >
> >> git log $hash..HEAD
> >
> > "HEAD" is just a git nickname for "whatever you have currently
> checked-out" (which can be a branch, a tag, or "detached" commit SHA ID).
>
> Mathieu mentioned: git log $foo_hash...branch_name
>
> That was the first time I've seen that used. All previous suggestions were
> HEAD, not branch_name. But they are all the same in this context?
>
> >> * know which branch each of those commits was on (e.g. master,
> branches/2020Q4)
> >
> > Unfortunately you'll find most normal git advice to be a bit frustrating
> with the FreeBSD repos, because FreeBSD doesn't work the way most people
> use git.  Specifically, the FreeBSD project does not ever merge branches
> (in the git sense of the word "merge").  Things would be very, very much
> easier if the FreeBSD project were to use git-style merging.  I believe
> there are discussions underway about adjusting the whole MFC process for
> the git world.  I admit that part of my motivation in writing this message
> is to provide grist for that mill.
> >
> > Fortunately even without git-merged branches, there are still git tools
> that help, though they're not as precise as one would like.
> >
> > Let's look at a concrete example with the beta ports git repo (which I
> just cloned), and compare the 2020Q4 and main branches.  I'll start with
> some overall exploration, then address your specific question.
> >
> > There are 298 commits in the 2020Q4 branch.  I know this because
> >    git merge-base origin/main origin/branches/2020Q4
> > tells me where 2020Q4 branched off of main: commit 5dbe4e5f775ea2.  And
> >    git rev-list 5dbe4e5f775ea2..origin/branches/2020Q4 | wc -l
> > says "299".  (The "rev-list" command is a bare-bones version of "log"
> that only lists commit SHA IDs.)
>
> [examples snipped]
>
> I followed that.
>
> I took the merge information as background and good-to-know, because
> FreshPorts won't be doing any merging. It just needs a good "git checkout"
> working copy.
>
> Sorry for such a long reply.
>
>
> * How FreshPorts extracts data
>
> FreshPorts is only interested in a snapshot of the repo with respect to a
> given commit.  It works on the 'repo as a whole' to extract values from the
> ports which were affected by that commit. Case in point: a commit to a
> parent port might affect any or all of the child ports.  All the child
> ports need to be refreshed.
>
> I am quickly concluding that FreshPorts must decide in advance what git
> branches it will pay attention to. At present, it follows all branches.
>
>
> * FreshPorts (without git) uses email to create XML
>
> When moving FreshPorts from subversion to git, one of the goals was to
> avoid relying on email to know that a commit has occurred. That is how
> FreshPorts has always worked. The email (from the CVS commit) was parsed
> and XML created. This code was updated for SVN. The XML is then used to
> load the commit into the FreshPorts database which then drives the website
> contents.
>
> When I started the GIT conversion, there was no commit email. "git log
> $foo_hash...HEAD" is how FreshPorts knows what commits to process.
>
> One positive aspect of email approach: it identified the branch. So far, I
> can't see how I can process the repo as a whole and see every commit and
> know what branch it was on.
>
>
> * Polling git
>
> It is beginning to sound like the FreshPorts git code for detecting
> incoming commits will be:
>
> Every N minutes, do this:
>
> for each repo in REPOS
>   for branch in BRANCHES
>     cd to the directory for that repo
>     git checkout branch
>     git log $branch_last_hash...HEAD
>     for each of those commits
>        process the commit
>     end for
>   end for
> end for
>
> At present, the REPOS and BRANCHES are:
>
> * freebsd BRANCHES="master branches/2020Q4 branches/2020Q3 branches/2020Q2
> ...etc"
> * freebsd-ports BRANCHES="master stable/12 stable/11"
> * freebsd-doc BRANCHES="master"
>
> Some might ask:
>
> * Why not just master and latest-quarterly for freebsd-ports?
> * Because commits to older branches sometimes occur (or at least I thought
> I saw one once)
>
> Commit hooks might also help, but I'm not sure if that will make things
> easier or complicate everything
>
>
> * When new branches arrive
>
> It is vital that FreshPorts remain automated as much as it can be. At
> present, under SVN, I might have to fix things perhaps 5 or 6 times a year,
> usually because a commit did not get processed.
>
> Keeping that in mind, I do not yet know how to handle the following
> situations:
>
> A new branch comes out.
>
> * Automation might be possible for ports quarterly branches
> * FreshPorts has to know there is a new branch
> * BRANCHES needs to be updated
> * I don't see that it can be automated for stable/*
> * need to handle 'git checkout branch' when branch does not exist?
> * Once branch exists, how do you find out about the commits when you have
> no starting point for 'git log'?
>
> Right now, a new quarterly branch is noticed when the first commit email
> comes through.  FreshPorts then does an 'svn co' for that branch.
>
>
>
> I'm hoping someone has good ideas for my edge cases.

Try the 'git branch -rv' command, cache its output in a file, and select
the lines with diffed to the new run of previous one.

>
> Thank you.
>
> —
> Dan Langille
> http://langille.org/
>
>
>
>
> _______________________________________________
> freebsd-git at freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-git
> To unsubscribe, send any mail to "freebsd-git-unsubscribe at freebsd.org"
>