Improving the handling of PR:s
韓家標 Bill Hacker
askbill at conducive.net
Fri Jan 11 14:10:50 PST 2008
Mark Linimon wrote:
> On Fri, Jan 11, 2008 at 07:35:41PM +0100, Peter Schuller wrote:
>> But understandable or not, the problem becomes particularly frustrating when
>> it affects PR:s that contain patches. Such PR:s constitute direct
>> contributions to the project. In cases where such patches are correct/good
>> enough, the non-application of those patches have what I believe to be
>> significant effects.
> I think you make some excellent points -- especially with how
> understandable it is for someone who's submitted a patch which got
> ignored, to not do the work to submit the next one.
> With respect to patches, the two things that I've tried -- and they have
> been insufficient -- are the following:
> - weekly email of "PRs containing patches"
> - weekly email of "PRs recommended by the bugbuster team"
> We can make the following observations:
> - the "push" paradigm doesn't work particularly well. Partly this is
> due to fatigue if you try to read through all this stuff (see below).
> - the "recommended" list has been slightly effective, but it's going
> to take some time for it to take off. For one thing, it has been
> underpublicized; for another, we don't have the culture of committers
> getting "warm fuzzies" from committing PRs. (Since no one working
> on that kind of stuff gets paid AFAIK, that's the only positive
> feedback they're going to get.)
> Last week someone made the observation that if these things were instead
> updated daily and posted to a web page, that it would be far more useful.
> I've taken that up as a task.
Observation: PR or 'other' - it is presently rather difficult to find
[older | most] patches- which reduces the opportunity for testing and
feedback, and increases the likelihood of having the situation reported
yet-again. And again..
- Can there be a port category (real or 'virtuaized', added - along the
'patches and workarounds'
With caveats, of course.... but hopefully also with provision for
co-located feeback, both positive and negative.
> Your other point about "latest PRs" is also a good one. That should be
> relatively easy to do. I'll take up that task.
Question: Can an improved 'data mining' project add value?
One wherein the tools to look for commonality - sometimes of the less
obvious nature - would be easier for all-hands?
I include in this the ability to build 'reports' - not just do the
present ad hoc searches.
If such could be done 'better', it might also benefit from crossproject
(i.e all the *BSD's) scope.
>> * The committer may not be familiar with the code and, even though the patch
>> looks good generally, it is felt that someone familiar with that part of the
>> system needs to have a look.
> In particular, this hurts for areas of the system that are unmaintained.
>> What I suggest is a system where a group of non-committers systematically
>> pre-process PR:s, to provide feedback and additional quality assurance to the
>> committer that is to process the PR.
>> This group of people could either be a self-proclaimed group of volunteers, or
>> perhaps a group of people satisfying the criteria of "guy we kinda trust to
>> do testing and provide a useful indication of sanity and correctness, but not
>> with a commit bit".
> So far it hasn't happened. We've set up the freebsd-bugbusters@ mailing
> list and the #freebsd-bugbusters IRC channel on EFNet (and please join
> us!) and the latter is where our last 2 bugathons took place.
> It's clear that there are several people who want to help process the
> PRs, and we don't have a good answer for them on "how can I contribute?".
> The existing tool, and social conventions, don't allow for non-committers
> to change PR states. As far as we've done in the past is to grant people
> "GNATS access" rights but not "commit rights", on an experimental basis.
> We've done this twice, and although it has worked well, just two people
> isn't enough. (One has gone on to become a full committer -- which is
> great!; the other current does not have as much time for FreeBSD work).
> Several hundred PRs were dealt with by these two folks, so I consider the
> experiment a success.
> What we used as a qualification was "track record of responding to PRs and
> questions on mailing lists", fwiw.
>> One possible answer to this that I have gotten in the past with another
>> project, is that noone is stopping me from just grabbing a bunch of PR:s and
>> posting follow-ups saying I've tested something, or otherwise giving
> Yes, but that's open-loop as well. It's the same situation as with submitting
> the patch in the first place: it's going to get frustrating if no committer
> picks it up and commits it.
> There's not too many people so thick-skinned and persistent as to keep
> going forward when no one's using their work.
>> For example, patches with a high confidence of correctness due to many people
>> affirming that they have tested it could be automatically prioritized for
>> committers to deal with, and there will be a clear and systematic record of
>> what testing was done, and by who.
> Right. The "priority" field in GNATS has been so abused as to become
> useless. (People assume that setting it higher will cause some kind of
> magic to happen.)
> My current thinking is that what committers ought to see is a metric of
> correctness, and a metric of priority, _as set by someone who's vetted
> the PR_. The weekly mailings are too poor an approximation of either
> to be useful.
> Adding the second metric would cure one problem that you don't mention --
> which is that few people have the interest and patience to plow through
> N-thousand PRs. It's not humanly possible to look at them all -- even
> the new ones as they come in. There's simply too many. So, you create
> an expectation "why bother, there's so many anyways". We need to break
> that chain of expectation. A good fix is a good fix. The PR count will
> never get to zero; I (with bugmaster hat on) would be thrilled if we can
> get to the point of just steady-state.
DB Admins, SysAdmins, 'Management' need BSD, too and not all the work
that needs to be done requires 'C' expertise.
Some of us also have spare bandwidth and hardware as well as time, so...
>> When I talk about priority above, I do not mean to imply that any committer
>> should work on things they do not want to work on. But I have to assume that
>> a lot of patches get committed because a committer decided to go and pick a
>> few PR:s to process, rather than the committer necessarily has a burning
>> interest in that particular patch.
> I think most get committed because a committer sees a PR come in on the
> mailing list and grabs it. Much less often do committers go through the
> database looking for things to fix. Again, the lousy "search/browse"
> capabilities of the existing tool let us down here.
>> As such, if said committer could spend those <n> minutes closing five times as
>> many PR:s because the up-front confidence in the correctness of the patch is
>> so much higher, perhaps it would help the system to "scale", moving some of
>> the burden off the committers.
> Right. What we need is to start small and create a _culture_ of where
> it's fun (or at least intellectually interesting) to work on this stuff.
> I think the IRC channel may still be the best bet for this.
> Once I finish fiddling around with the sparc64-6 and sparc64-7 release
> builds (the first approximation is finished, but I believe we can still
> add some more packages), I intend to task-switch over to looking towards
> defining a PR workflow and floating some proposals. I hope to have
> something concrete to present at BSDCan. Please join us on bugbusters@
> to discuss any more ideas, too.
Coders will always set their own priorities. Even when working for a wage.
But all-hands benefit from greater stability and fewer 'critical' things
breaking, so rational self-interest might be more effectively applied by
each participant if the 'big picture' - in several flavors - was simply
published in some 'better' manner.
You've ID'ed some of the possibilities - there are bound to be more.
Perhaps some of the committers - or at least the project in general -
could benefit from more 'virtual office staff' - folks who specifically
are NOT coders - to remove obstacles and help leverage their expertise?
Background research, documentation, finding and enlisting testers who
can help, building and maintaining databases - *doing* tests. Some of
that is more amenable to being 'directed' than the coding efforts
themselves. Different personality types...
More information about the freebsd-current