Policy on closing bugs

[Kubilay Kocak]

On 24/05/2019 9:30 pm, Grzegorz Junka wrote:
> 
> On 24/05/2019 11:12, Kubilay Kocak wrote:
>> On 24/05/2019 8:07 pm, Grzegorz Junka wrote:
>>> Hey,
>>>
>>> Is there any policy/document when a bug can be closed? For example, 
>>> is it OK to close a bug that is fixed upstream but not yet in ports?
>>>
>>> Thanks
>>> GrzegorzJ
>>>
>>
>> Hi Grzegorz,
>>
>> Bugs are closed after they are "resolved". Resolved means a resolution 
>> has "occurred", but more precisely, the "thing reported" has been 
>> resolved. Resolved doesn't necessary mean "fixed" (see below)
>>
>> What resolution is appropriate/set depends on the context of the 
>> issue, usually the specific nature of the request/proposal. Is there a 
>> specific bug you're referring to? I can speak to that case 
>> specifically if so.
>>
>> For example however, if the bug was a "bug report for the 
>> port/package", fixed upstream hasn't fixed the port, so not usually, 
>> no, that wouldn't be considered sufficient to be "resolved" and closed.
>>
>> Usually commits upstream are backported to the ports, and they are 
>> closed when those are committed.
>>
>> There can't be policies for this perse, as its completely 
>> context/request dependent.
>>
>> Resolutions can take place either by way of:
>>
>> 1) A change is made: a commit, usually, but could be a wiki update, or 
>> a DNS update for infrastructure changes, etc.
>> 2) One of the 'non-change' resolutions: not accepted, unable to 
>> reproduce, feedback timeout, etc
>>
>> Nothing about the above is special or different than most other issue 
>> trackers (generally speaking).
>>
>> Regarding states, we have New, Open, In Progress, Closed
>>
>> New: Not touched/Untriaged
>> Open: Initially Triaged (classified)
>> In Progress: Has a real (person) Assignee, action has started
>> Closed: Change(s) Made, OR "Non-Change" resolution set.
>>
>> There's nothing special/different about these either, except that we 
>> like to have a real person assigned before in progress, and before close.
>>
>> Happy to answer any more questions regarding issue tracking, etc anytime
>>
> 
> Hi Kubilay,
> 
> Thank you for a detailed response. Yes, this is related to a particular 
> defect. I didn't mention it because I didn't want to be picky and seen 
> as causing troubles :) Also wasn't sure what's OK and what's not. Here 
> is the defect:

My pleasure. I understand the desire not to want "cause trouble", but 
it's also important that everyone feel comfortable asking questions, and 
understand/clarify how things works (or should work, ideally). We need 
to see more of it, not less.

> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=238086
> 
> I appreciate Yuri's contributions to the community and my intention 
> isn't to bring this up for judgment. Even though as a FreeBSD user I 
> might feel a bit ignored and shuffled under the carpet after the defect 
> has been closed, my intention was more to find out if maybe a new state 
> "Postponed" could be added for a defect in a state like this one?
> 
> GrzegorzJ

So there's a few issues involved, that are worth making very distinct:

- A FreeBSD port/package and its users are affected
- Upstream has apparently fixed the issue, but there's not much detail 
about how, where, when, etc
- The bugs resolution type

We'll run through those individually so its hopefully more clear how 
they might interact/overlap with each other:

1) If a port/package is broken in some way, we want to fix it, and as 
maintainers, we have signed up to do that. This is not controversial.

For users, its not always (I would argue in most cases), possible or 
easy for them to distinguish between a port problem, and a software 
problem, and who (freebsd or upstream) should be primarily responsible 
to a) get the initial bug report b) fix it in the first instance. I 
personally don't believe users should be expected to know or do this, 
but its great if/when they can.

There are arguments on both sides of the (unfortunate) 
upstream/downstream divide, about users reporting bugs to the "wrong place".

Sometimes downstreams hack software to make it work or do things 
differently in their distribution/OS, and sometimes these things break, 
and upstreams get the report.

Sometimes upstreams break things, and downstreams get the reports.

It's a difficult problem to solve completely and permanently, so 
ultimately it's the relationship between downstreams/upstreams that is 
the most important thing to cultivate.

Having said that, at least in the former case, I don't think its too 
much of a burden for us to receive reports and close them (where 
appropriate) as "wont fix / not accepted" after commenting that the 
report should go upstream.

The question as to what and when is appropriate is very case dependent, 
but the minimum (in my opinion) is that it should be explicitly clear to 
the reporter and documented what the complete rationale, analysis is to 
resolving in that manner.

2) If an upstream has fixed an issue, all else being equal, we ought to 
be motivated to identify the specific upstream fix/commit/pr/issue, and 
look to include it in the port, if possible without a version update. In 
particular, we should do this so that the fix can be merged to the 
quarterly (security and bugfix branch), which doesn't take version 
(feature) updates. Bugfixes and security updates is the promise of 
quarterly, if we wait for version/feature updates to get bugfixes, 
quarterly doesn't get them.

It's *very* helpful if users can help identify the specific upstream 
references, but it's definitely not a requirement.

Note also that bugs can and should be re-opened by users *at any time* 
if additional information comes to hand that precludes or updates the 
last 'resolution' at last close. This does not mean that they should be 
re-opened spuriously, or because you don't like the decision personally.

3) The resolution in this case "not a bug" is not the most correct for 
the apparent resolution "wait for upstream update". It is a bug, and it 
has (apparently?) been fixed upstream, and a freebsd port is currently 
impacted by it.

Implicit in a bug report "port foo is broken when bar" is "and the bug 
should be fixed in the port". If a user included a patch to fix it, and 
the maintainer declined the change, the bug would be closed "not accepted".

That there exists some commit upstream that fixed the issue, means the 
most correct resolution (of the ones we have today) would be 'Not 
Accepted'. Its only slightly not quite "not accepted" because an 
upstream commit/fix was not identified (or was, but it wasn't commented on).

Side Note: I recently changed the resolution name "Rejected" to "Not 
Accepted" for obvious reasons, though I can now see that this still 
needs to be improved, to cover the case where a 'change/patch' hasn't 
been offered. I had considered "Not Accepted / WONT FIX", but that was 
too long. I'll think about this some more.

Very very very few bugs are closed "talk to upstream" or "wait for a 
version update", and for those that are, its usually very clear that 
upstream is the "better" place for the report, or there are other issues 
precluding backporting a fix.

In this specific case, it would be great to have someone identify the 
fix concerned upstream, re-open the bug with those links/references, and 
explicitly request that they be included in the port. That's already 
what happens in the vast majority of cases, even to the extent of 
maintainers creating bug reports/PR's on our users behalf.

Hope that helps GrzegorzJ!

If you or anyone else is interested in the subject, don't hesitate to 
ping me (us, bugmeister) on IRC (#freebsd-bugs / freenode) to talk more 
about issue tracking, productivity, problems, improvements, policies, etc :)

./koobs