Bug Triaging

This page has not been reviewed by our documentation team (more info).

1 Little Guide to Bug Triage
2 What to do

Little Guide to Bug Triage

Basic Triage: No particular skills required, just a strong ability to read for comprehension, and familiarity with MusicBrainz.

Ask yourself these questions:

Has this bug been reported before?
Does this bug have enough information for the developers, and does it make sense?
Is this bug filed in the right place (check categorisation)?
Is this bug's priorities and severity sensibly set?

What to do

Has this bug been reported before?

Merge open reports for the same issue.
TODO: write about merging and retesting when one of the reports is closed (check it's really the same problem, not another with a similar effect, etc)

Does this bug have enough information for the developers, and does it make sense?

Bug reporting is actually pretty hard, and not everyone is good at it, or cares about it. Sometimes people use bug reports as a place to unload frustration over a problem they are facing. Much more often, excellent bug reports are hidden under titles like "it's broken", or simply explained unclearly. A good bug report needs to have most of the following:

A clear and concise summary in the title, explaining what the problem is.
A more complete explanation of the problem. A great bug report will read like this:
- What the user was attempting to do when the problem happened
- What the user expected to happen
- Precisely what the user did in order to make it happen
- What really happened instead
- Whether the bug is reproducible if these steps are repeated
- A brief rundown on the users environment (for picard, that might be os, python version, method of installation. For MB, that might be browser version, if things like greasemonkey scripts are running, connection speed if it seems relevant)

Ideally, independent confirmation from at least a second person that a bug is reproducible. Or that it's not, by repeating the exact same steps.

So what can you as triager do here?.

Rewrite summary tests so they accurately describe the problem.
Clarify the steps listed above with users, help them explain what happened.
Test if the problem is reproducible. Test it on both the live server, if it's not dangerous to data or something you can cancel out of, and on the test server.

Is this bug categorised correctly?

A frustrated person faced with an unfamiliar interface (and we would hope the bug tracker is fairly unfamliar to most people) will often not closely read the text on screen, or set correctly all the options available to them. Before you're done with a bug, check that it's assigned to the right categories, and reassign if you need to.

Is this bug's severities and priorities set correctly?

If in doubt, don't touch. However, as a rule of thumb, people fall into two categories: Those who don't care or notice the severity fields, and leave them on defaults, and those who turn them all the way up to as high as they can go. Most bugs are not urgent, or severe, and certainly not both, yet you'll see states like that set on things like a missing apostrophe on a page few people see. It's quite ok to turn those down. Second rule of thumb: It's pretty rare you need to turn _up_ a bug severity. Let developers do that themselves.

Discuss: Do we have a defined set of criteria to define "severe" or "critical" (or whatever it is trac uses, I didn't check yet, will before cleaning this up though) Personally,

Older notes

The start of some notes about BugTriaging (see the GreatDispute)

Spot early which are bugs ("It doesn't work as intended") and which are feature requests ("It doesn't work how I expect/It doesn't work how I want"). Is it an HCI/UI bug? ("It doesn't work how I expect, but does work exactly as intended")
Weed out duplicates.
Prioritising. There can be several scales of priorities, and they can be quite orthogonal to each other, sometimes even completely in opposition. We should be able to rate on various scales, such as:
- in order of urgency (not necessarily importance - some things are urgent, but not very important, and some things are very important, but not at all urgent. That's a bit difficult to grok sometimes):
  1. Data is lost (Existing data is damaged or removed, when it shouldn't be). I guess MB doesn't have much possibility of this. Picard may though. Includes "CRASH", if it causes damage to existing data.
  2. User time is lost (Data entry needs to be done over, for instance) Includes "CRASH" if it means you have to redo your tagging.
  3. Data is duplicated because... (bad, but not as bad as a crash making you reenter a whole album)
  4. Interaction: Does this problem cause a chain reaction? Is this potentially related to other problems?

- in order of importance
  1. Scale of problem (does it affect a thousand edits a day? or one?)
  2. Scale of cleanup (if this isn't fixed in a day or a week or a month, how many edits will need to be reentered? Corrected? removed?)
  3. Scale of fix (hard to judge from outside, but is it a huge change, database schema, or is it a one liner, will it cause an interaction)
  4. Can it be worked around? If there's a known and doable workaround, can we educate people to use it in the mean time?

Other things:

Is it reproducible on the test server (if yes, what specifically and exactly are the steps to reproduce it)
Is there more than one bug here? If yes, they need splitting

... and more, when I get around to it.

NeedsIntertwingling NeedsCategorizing - started by KrazyKiwi