PR backlog [was: Re: usb/umass, devfs: this sucks]

[Mark Linimon]

On Wed, Dec 26, 2007 at 06:15:16PM +0100, Henrik Gulbrandsen wrote:
> Fixing and merging are good, but it seems to me (as an occasional patch
> contributor without commit privileges) that the bottleneck for USB is
> still in the handling of incoming patches [...] if a one-line fix
> such as that in usb/78984 has not been applied after more than a year,
> how long will it take for patches that involve multiple subsystems?

I'll put on my bugmeister hat for a moment.

First, I share your frustration.

Second, unfortunately, it's not just USB.  We suffer from this problem in
several other areas, notably, patches for the userland utilities ("bin").

There are two interrelated problems that create a chicken-and-egg
situation:

 - the PR tool is insufficient for our needs;

 - there's not a "culture" of going and fixing bugs outside of the code
   one usually works on.

As for 1), once the releases are out, I intend to start working on defining
what "our needs" are.  As I've stated before in other places, until we
understand those, and get community buy-in to define an actual "process"
rather than just a particular PR system, it's unwise just to change the
PR system.  (IMHO, there's no reason to further automate a process that
doesn't work correctly.)  I hope to have something concrete to present at
BSDCan in May.

As for 2), I've scratched my head for what to do about that for a while,
and not been able to make much progress.  Here's what we've tried:

 - The creation of a weekly posting "bugs the bugmeister team thinks are
   ready for commit".  This doesn't seem to have attracted the desired
   attention.  Perhaps this is a problem of "push" not being the right
   solution here; perhaps it just hasn't been publicized enough.

 - The creation of a hack for classifying PRs, the "[tag]" convention.
   This is simply working around the weakness of the tool.  However,
   it is sufficient to be able to generate weekly email sorting the
   PRs by tag, and another email showing only PRs with patches, also
   sorted by tag.  If you are a committer, it's also possible to run
   queries via:

   ~gnats/tools/showwithtag <tagname>

   to get a summary.

 - Trying to get more traffic on the freebsd-bugbusters@ mailing list.

 - Trying to create some interest in #freebsd-bugbusters on EFNet on IRC.
   This has not attracted enough committers to be viable yet.

 - Holding some "bugathons" (idea stolen from NetBSD) where we try to get
   committers to come onto the IRC channel at particular weekends to try
   to interactively work through PRs.  This had some success, but we have
   not done one in a while.

The odd thing is that for ports, the existing PR system -- plus, most
importantly, the hacks we've added on top of it -- works reasonably well.

 - Each port has an explicit "maintainer" field (even though many of
   the entries are null).  Most of the src codebase does not, therefore
   no one in particular "owns" it.

 - We've taken advantage of that to layer a PR auto-assigner on top, that
   also sets things to 'feedback'.

 - portsmon is also able to track PRs by the port they affect, and semi-
   weekly reminder emails are sent out.

But the first of these items is really particular to ports.  Also, more
ports PRs than src PRs are upgrades/bugfixes, rather than true bug reports
that need substantial investigation (in fact, the ratio is probably
exactly reversed).  This means we can clear a proportionally larger number
of ports PRs.  All of this has helped get the port PR count down over the
last several years, to the point where it no longer seems as overwhelming
as the backlog in the other areas.  The size of the backlog creates a
substantial disincentive to try to fix a handful -- thus perpeturating
the cycle.

What we all need to understand is that the PR count will never be at zero;
if we can instead settle for a steady-state, where the most concrete PRs
can be worked on as they arrive, then I'll feel we've have made great
progress.

Unfortunately I don't have any brilliant insights as to how to make the
work more interesting; most of my ideas have the focus of simply making
it less frustrating.

I'll throw the floor open for brainstorming at this point.

mcl