Handling Bug Reports

by Charles Miller on June 9, 2003

mpt linked to Dave Nichols' article: I'd like to complain about this software, which contains the following wisdom about accepting bug reports:

The net effect of the lack of easy feedback channels is that the average user feels a sense of frustration and powerlessness. They get really irritated by their software, and no-one is listening. At least when you enter a Bugzilla bug you feel if(sic) you have done something constructive. Maybe, just maybe, someone sometime in the future will experience a better interaction thanks to your report.

This rings true with me. I like having the option of giving a software producer my opinion on what I want their software to do, and how. A while back, I reported a bug to Apple, asking for a way to synchronise Safari bookmarks over iSync. In the last Apple update, that feature was included. That made me feel pretty good about having submitted the bug, even if my submission had absolutely no impact on the schedule whatsoever.

On the other hand; looking at Bugzilla or any public bug-reporting forum for a popular product (my favourite used to be the bug reporting site for IBM's Visual Age for Java) reveals a down-side to having a public bug-reporting mechanism:

  • The priorities of the people who make the software (let's call them developers) will never match the priorities of any individual bug reporter (users)
  • With sufficient bug reporters, certain bugs that are not on the developers' radar, for various reasons, will be nominated by a significant mass of users
  • Bugs that are on both the users' and developers' radar will be fixed promptly, and have little attention paid to them
  • Thus, certain low-priority bugs will end up being the ones that attract the most user comments, and the most user votes

This leads to a dilemma. If the software writers do not have the courage of their convictions, they will waste time fixing bugs that should be low priority, but that attract the attention of a vocal minority: (Linux or Mac users, for example1). If the writers stick to their guns, they will be feeding a public resentment among users that will spill over into newsgroups, user groups, Slashdot, you name it. The “This is the most voted-on, commented-on bug, why isn't it fixed!” syndrome comes into play.

So, with that rather lengthy preamble out of the way, here is my list of pre-requisites for the public bug-reporting system that doesn't suck as much.

  1. Allow any user of the product to submit a bug.
  2. Every bug is given a priority by the people actually running the project. This priority is communicated back to the user as soon as possible, so they know they have at least been heard. Feel free to wrap these priorities in legal weasel-words. If it is an open-source project, mention that (aside from the last case) the user may implement it themselves instead. It is important to be honest, and prompt. People like to know that their bug-report has at least been read and evaluated by someone.
    • We consider this a high priority and will attempt to implement it in the very near future.
    • We consider this a high priority, but due to the size of the task, it may have to wait a few releases.
    • We do not consider this a high priority because [foo]. We want to implement it, but don't hold your breath.
    • We do not wish to implement this, ever, because [foo]. We apologise for not meeting your particular needs, and suggest you [use another product that more suits your need / fork the codebase].
  3. Do not allow public comments or votes on issues. If you can trust your developers to stay on topic, then technical comments (i.e. not about the prioritisation of bugs) are still a good idea, but discussions on the merit of particular issues are pretty much worthless.
  4. Allow comments on the merits of an issue, but swallow them. If the commenter is new to the issue, treat it like a new bug report, and give them the same feedback. If it's a repeat, inform the user that nothing has changed since they last showed interest.
  5. Allow public searching of the issue database, and the current status of each issue. This at least allows people to point to known problems.
  6. Inform bug reporters when their bug is fixed. It's one of those personal touches that makes people feel happy for having taken their part.

The idea here is to give users empowerment equal to their station. I know this could be a controversial point, but ultimately the development of an application is the responsibility of the people who are actually developing it. This is true whether it is closed- or open-source. If an application doesn't listen to the needs of its users, that application will fail. What are the needs of the users, however, are up to the developers to determine from all the evidence, and a bug-tracking database is a very bad measure of what is important across the whole range of users.

1 The author of this article is typing it on his kick-ass flat-screen iMac, and will post it through the Linux box that is providing his net connection, email, and so on. So bugger off.

Previously: About a Dead Rat

Next: Bad Thoughts