Archive for the 'Bug tracking' Category

Catching regressions faster

Wednesday, February 10th, 2010

My latest post on the Mozilla Security Blog:

Fixing security holes without introducing new bugs

Shorter URLs for bugzilla.mozilla.org

Thursday, January 14th, 2010

Reed Loden set up bugzil.la as a redirect for Bugzilla bugs and searches. Examples:

Buglist in input order

Thursday, October 15th, 2009

My newest Greasemonkey script, Buglist in input order, lets you sort a bug list according to an order external to Bugzilla.

For example, you can view the list of bugs related to topcrashes sorted by how common each crash is. First, load the list of Firefox 3.5.3 topcrashes. Second, use the "collect buglinks" bookmarklet to view a bug list consisting of the bug numbers listed there. Third, click the "#" added by this Greasemonkey script to sort the bugs by their order on the topcrash page.

Hopefully, this script will allow us to avoid duplicating information between Bugzilla and CrashKill wiki pages, and thus reduce the risk of information in both places becoming out of date.

Language barriers and bugs

Saturday, July 18th, 2009

A text-input bug affecting many Arabic speakers was discovered when we were about make Firefox 3.5 release candidates, and we chose to include a fix in Firefox 3.5.1 rather than delay Firefox 3.5.

The zero-day vulnerability in Firefox 3.5 was a frequent crash on the front page of a top-500 Russian site, but we didn't find out about it until after we shipped Firefox 3.5.

The reason we found out about these bugs late isn't a lack of non-English beta users. 1.8% of English Firefox users are beta users, and 1.6% of Russian Firefox users are beta users. Arabic beta use is much higher, at 3.5%, perhaps because Firefox 3 was such a large improvement over Firefox 2 for rendering Arabic text.

Instead, the reason may be that people who don't know English can't report bugs easily. They can't even search Bugzilla to find out whether their bug is known. They went out of their way to try betas, but beta-testing alone does not keep major problems from making their way into a release.

How can we enable the rest of the world to participate in reporting bugs, or at least ensure that we find out about the most frequent problems in other languages?

New guide to triaging crash bugs

Monday, June 29th, 2009

Last night I posted a new guide to triaging crash bugs.

Reports of crash bugs require a variety of skills to turn into useful bugs: knowing common support issues, tactfully interacting with bug reporters, reducing testcases, finding regression ranges, and knowing how to get each bug to the correct developer(s). I don't expect every triager to be a guru at all of these, so my guide incorporates a workflow that should allow triagers with differing skills to work together efficiently.

I'm going to work with the QA team to organize some "crash bug days" on IRC, but I'm interested in feedback before that as well.

Getting bugs done

Monday, April 20th, 2009

I believe Bugzilla's workflow can be improved using one of the central ideas from Getting Things Done, the "next action".

Currently, the answer to the question "what is needed to move this bug forward?" is scattered throughout each bug report. Sometimes it's a keyword, sometimes it's a review flag, and sometimes it's the third-to-last comment. It takes me maybe a minute per bug to determine whether I can help, and this wasted time adds up quickly.

Next-actions for bugs

I propose replacing the status field with a next action field, and the assignee field with a next action assignee field. The "next action" field answers the question "what is needed to move this bug forward":

All bugs
  • Understand what's wrong with the code
  • Write patch
  • Automate test
  • Review patch
  • Approve patch
  • Push patch to mozilla-central
Major refactorings
  • Create tests for existing code
  • Design new code
  • Review design
  • Test patch on try server
  • Test patch against fuzzers
  • Fix whatever caused the back-out
New features
  • Provide ideas
  • Experiment by writing extensions
  • Experiment in a usability lab
  • Consider security ramifications
  • Consider accessibility concerns
  • Decide whether it goes in
  • Specify desired behavior in detail
Layout bugs
  • Reduce testcase
  • Find regression range
  • Check against CSS spec
  • Get crash stack
  • Get hang profile
  • Run under valgrind
  • Debug with gdb

Organizing actions by context lets me remember projects when I can move them forward, rather than when they only increase my anxiety. This may be even more important in a community system: who you are is a key context determining whether you can do something.

With a next-action field, it would be easier to find places to put my skills to good use:

  • Did Brendan hope to have this patch fuzzed?
  • What bugs are waiting for input from the security team?
  • What bugs could benefit from extension-space experimentation?

With the addition of a next-action-assignee field, I'd also be able to do searches like:

  • What have I been asked to do in Bugzilla?
  • What blockers are stuck on me?
  • What bugs need someone to volunteer to make a reduced testcase?

Simplifying Bugzilla

I'm asking for new fields, but I think this will actually make Bugzilla less complicated. The "next action" field and its assignee would replace:

  • The current "status" field, which is mostly useless because each open state (unconfirmed / new / reopened / assigned) can mean so many different things.
  • The current "assignee" field, which is misleading when the next action required is anything other than "write a patch" or "check it in".
  • Eleven keywords: uiwanted, qawanted, helpwanted, stackwanted, icon, testtracker, crashreportid, regressionwindow-wanted, testcase-wanted, checkin-needed, push-needed.
  • Status whiteboard markers like "[needs review gavin]" and "DUPEME".
  • Many comments that are only temporarily meaningful.

It would also make our process more transparent: there is always a first-level answer to "Why hasn't this bug been fixed yet?"

Deconfirmed

Thursday, April 9th, 2009

Bug reports in bugzilla.mozilla.org can now be changed back to UNCONFIRMED. Sweet! I'll use this next time I accidentally file a bug that lacks a reduced testcase a "NEW".

(From one of the three long threads on mozilla.dev.planning about Bugzilla statuses and workflow.)

TidyBug

Thursday, February 26th, 2009

My newest Greasemonkey script, TidyBug, cleans up the show-bug page on bugzilla.mozilla.org.

TidyBug hides empty fields, making it easier to see the fields that are populated and reducing the amount of scrolling needed to see the comments. In the example above, TidyBug removed 61% of the space above the first comment (from 985 pixels to 381 pixels). Even on busy bugs, such as upvar2, TidyBug makes the bug considerably cleaner (upvar2 screenshot).

Some other things TidyBug does:

  • Makes form control borders only appear on hover to reduce visual noise.
  • Hides four fields that are mostly useless in bmo (version, architecture, OS, severity).
  • Adds keyboard shortcuts for accessing hidden fields and several common actions.
  • Clicks the "hide obsolete attachments" link for you.

Is this useful for many people? Should parts of it be uplifted into bugzilla.mozilla.org or the Bugzilla distribution? What new keyboard shortcuts or other features would you like to see?