Improving the handling of PR:s
Peter Schuller
peter.schuller at infidyne.com
Fri Jan 11 10:35:37 PST 2008
From the hideous flamewar-about-everything thread:
> We clearly have a disconnect on PRs. More come in than we have any way
> to handle. This is clearly most distressing when the PRs contain
> patches and/or test cases. Again, I'm open to ideas on how to set up
> something where more people can participate.
Eric Anderson posted a suggestion that I read and won't comment on other than
to say that from the perspective of the non-committer it sounds like a good
idea.
I have another suggestion.
The fundamental problem, I think everyone agrees, is that there is simply not
enough commit bit developer time available to attend to all the PR:s. This is
understandable.
But understandable or not, the problem becomes particularly frustrating when
it affects PR:s that contain patches. Such PR:s constitute direct
contributions to the project. In cases where such patches are correct/good
enough, the non-application of those patches have what I believe to be
significant effects.
The one perhaps lesser effect is that other people are bitten by the same bug
(which translates into loss of time on potential future contribotors) even
though a fix is available.
The second, possibly worse effect, is that the original submitter, I believe,
is less inclined to spend more time in the future contributing, if said
submitter feels the work is wasted because no one commits or even comments on
the PR.
In the case of PR:s with patches, why do they not get enough attention?
Obviously there is no technical reason why the application of the patch
itself should be a problem (that could be completely automated even). The
problem, presumably, will be that the time it takes to process the PR is much
more than the mechanical act of applying a patch because (and this list is
probably not exhaustive):
* The patch must be looked at and understood prior to application or even
testing. Does it comply with style guidelines? Does it break something else
as a side-effect? What are other implications? And so on.
* The committer may not have access to the hardware, or may not have a
software setup that allows for testing. This means doing such testing
suddenly requires a lot more effort.
* The committer may not be familiar with the code and, even though the patch
looks good generally, it is felt that someone familiar with that part of the
system needs to have a look. This means the committer can either spend
additional time on understanding the surrounding codebase, or leave it for
the subset of committers that have that understanding already.
One observation is that there is no realistic way for the community of
non-committers to help effectively the committer in the sense of lessening
the amount of work required to process the PR.
What I suggest is a system where a group of non-committers systematically
pre-process PR:s, to provide feedback and additional quality assurance to the
committer that is to process the PR.
This group of people could either be a self-proclaimed group of volunteers, or
perhaps a group of people satisfying the criteria of "guy we kinda trust to
do testing and provide a useful indication of sanity and correctness, but not
with a commit bit".
Given a fairly simple PR with a fairly simple patch, would not the work of the
committer be greatly lessened if there was a track record of the
above "pre-processors" shows that <n> people have performed building and
testing of the patched version on architectures/environments, <x, y and z>,
and that they generally like the patch (or that they don't)?
One possible answer to this that I have gotten in the past with another
project, is that noone is stopping me from just grabbing a bunch of PR:s and
posting follow-ups saying I've tested something, or otherwise giving
feedback.
The problem is that there is no real indication that this will cause anything
to happen other than generating more E-Mail traffic.
What I suggest may make the difference is to have a *system* for this.
Preferably with software to support it.
For example, patches with a high confidence of correctness due to many people
affirming that they have tested it could be automatically prioritized for
committers to deal with, and there will be a clear and systematic record of
what testing was done, and by who. And the individual contributor (for
testing) can really feel that the time spent was worthwhile, because the
feedback was taken by the system and used for actual effect.
When I talk about priority above, I do not mean to imply that any committer
should work on things they do not want to work on. But I have to assume that
a lot of patches get committed because a committer decided to go and pick a
few PR:s to process, rather than the committer necessarily has a burning
interest in that particular patch.
As such, if said committer could spend those <n> minutes closing five times as
many PR:s because the up-front confidence in the correctness of the patch is
so much higher, perhaps it would help the system to "scale", moving some of
the burden off the committers.
Perhaps one can either go so far as to have "full committers" give their
blessing for a particular patch, possibly based on feedback such as above,
but leave the act of actually commiting the patch and doing the actual smoke
testing (that it builds etc) to someone who does have a commit bit and is
trusted with it, but is not allowed to make any commits that are not directly
blessed by a "full" committer.
There is some precedent for similar systems. Squeak had the concept
of "harvesters". They seem to have changed things a bit nowadays, but
seemingly use a similar approach still (I am no longer up to date):
http://wiki.squeak.org/squeak/3152
(see the comment/review/vote period)
--
/ Peter Schuller
PGP userID: 0xE9758B7D or 'Peter Schuller <peter.schuller at infidyne.com>'
Key retrieval: Send an E-Mail to getpgpkey at scode.org
E-Mail: peter.schuller at infidyne.com Web: http://www.scode.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: This is a digitally signed message part.
Url : http://lists.freebsd.org/pipermail/freebsd-current/attachments/20080111/1f516393/attachment.pgp
More information about the freebsd-current
mailing list