Weekend PR smashing

[Mark Linimon]

[adding freebsd-bugbusters@ to the Cc:]

I'm sorry I didn't respond to your earlier message.  I am currently way
behind on tasks that I've already promised people.

The KDE howto that you cited (http://quality.kde.org/develop/howto/howtobugs.php)
is quite a good one.  Our pr-guidelines document isn't as thorough.

Part of the problem is that we don't have enough metadata in GNATS to
capture some of the things that we would like bugbusters to do, e.g.,
as they suggest:

 - attempting to reproduce an 'unconfirmed' bug and change it to 'new'

The closest that we have is the 'analyzed' state, which we have used in
the past to indicate 'confirmed'.  If you can indeed reproduce a bug,
it's fair enough to let bugmeister@ know and we can change the state.
Alternatively, you can submit a followup and suggest that.  Several
of the folks on the bugbusting team (e.g. people who have GNATS access)
monitor the mailing lists and try to track followups.  (I personally
track bugs@, ports-bugs@, and a few of the others, but not some of the
specialty ones like net at .)

 - check if a report is a duplicate

Reports that are duplicates indicate that various users are being affected
by one underlying problem.  At one point I was trying to gather them all
into a page.  I was hoping more people would do the analysis and send me
additions for it.  However, it looks as though the script that generates
that page has rotted.  I'll re-add it to my list of things-to-do ...

  http://people.freebsd.org/~linimon/studies/prs/well_known_prs.html

At this point it may be easier to look at the template that I grind up to
produce that page:

  http://people.freebsd.org/~linimon/wellknown.prs

It's clearly over a year stale, and could thus use a new set of eyes.

 - add '(regression)' to the title if indicated

I think we've done a fairly good job of getting '[regression]' (our styling
of the same idea) into the existing ones, and keeping up with new ones as
they come in.

Similarly, we've done pretty well with '[patch]' AFAIK.

That's my reaction to some specifics on the the KDE page.  Now for some
more general comments.

We have more kern/ PRs than any other category.  This category is
overloaded to mean both kernel, libraries, networking, and device
drivers.  http://people.freebsd.org/~linimon/studies/prs/pr_tag_index.html
makes this much more tractable.  (I have other ideas about how to do
much better, but see below.)  Whether or not existing PRs are still
relevant seems to vary a bit by sub-category.

There would also a slightly different way of looking at things, the ones
with '[panic]' in the Synopsis.  Hmm, I thought there was such a page,
but it doesn't seem so.  I'll put it on my list to create one.

As for the bin/ PRs, you'd be surprised at how many of them are still
relevant, even after several years.  I've resisted the suggestion that
others have made in the past just to close them for this reason.  (In
the past I have made an effort to contact submitters and ask them if
they are still experiencing the problem, but portmgr duties have kept
me away from that for a long time.  This is something else where we
need help.)

The i386 and amd64 PRs are more likely to be "can't install/run on a
particular piece of hardware", and generally require the most interaction
with users to isolate the problem (e.g. to ACPI interacting badly with
a buggy BIOS; irq problems; devices not being supported by our current
drivers; and so forth.)  To work with these people need to have a bit
of experience with low-level debuggers and how to examine stack traces.

> one of the things that concerns me is the sheer number of PRs with
> patches that either have been committed without the PRs being updated

Well, that's a task that needs more people working on.  You can start
by looking at:

  http://people.freebsd.org/~linimon/studies/prs/prs_possibly_committed.html

which is PRs that have had followups appended to their Audit-Trail by
the checkin scripts.  If these changes have already been merged to all
the relevant -stable branches, they should be closed; otherwise, if they
are not already set to 'patched', then they should be, and assigned to
the committer who made the change.

> or the patches are simply sitting idly in PRs.

There are certainly a large number of these.  IWBNI we could get more
of them into the 'analyzed' state to claim 'we think this patch might
solve the problem'.

> The list by the bugbusters waiting for committers to check them out
> is pretty huge as well:

Yes, I'm well aware of that, since I set it up :-)

This goes to the more general problem that it's more fun to add new things
than it is to fix existing things.  FreeBSD has traditionally done much
better at adding new features than in the more mundane maintenance work.
This is a problem not just with FreeBSD or even open source projects in
general, but all software.

What's interesting is that this situation is slowly changing (again IMHO).
Over the past year we have seen several people 'graduate' from being on
the bugbusting team to having src commit bits.  Slowly some of the PRs
that fit the classical "user-annoyance" classification are being fixed.
As more people start to do this, the more tractable the problem starts to
seem, even for long-time committers who may have despaired over the state
of the database over the years.

(It is also worth mentioning that with the help of Remko, Gavin, and
others, the Synopsis lines of the existing PRs are now much more likely
to be useful and correct than even, say, two years ago; and thus, the
data-harvesting pages I put up became possible.)

Building up people's belief that the overall problem is tractable is a
chicken-and-egg problem.  Again, compared to where we were, a great deal
of progress has been made, albeit slowly.

> It's a little difficult to muster up too much motivation to fix issues when
> the fixes will then sit waiting for committer review for months on end.

My suggestion is to avoid looking at the overall number of PRs, and instead
find things that interest you to work on.  Are you interested in helping to
triage existing PRs; or perhaps, in responding to new PRs that come in and
work with users to try to narrow down the problem (especially useful for
install/boot issues and regressions); or perhaps in one particular part of
the system such as networking?  Or perhaps the "overall user experience"
things such as sysinstall?

> Without annoying committers, is there any way I can help get patches
> "through" and into the tree in less than a lunar cycle? ;)

The only way that I have found so far is to create the specialized reports
(both HTML and email) and try to publicize them.  New ideas are welcome.

Lastly, my (currently stalled) project to create a prototype for a new
PR system tries to deal with some of the limitations of, and frustrations
with, GNATS.  For instance, see my presentation at BSDCan 2008:

  http://people.freebsd.org/~linimon/presentations/prs2008/

But I'll reiterate, the way to go (IMHO) is to pick one thing out of the
"my suggestion" paragraph above that you think might be interested in, and
start with that.

I'll be happy to listen to any feedback you might have.

mcl