New guide to triaging crash bugs
Last night I posted a new guide to triaging crash bugs.
Reports of crash bugs require a variety of skills to turn into useful bugs: knowing common support issues, tactfully interacting with bug reporters, reducing testcases, finding regression ranges, and knowing how to get each bug to the correct developer(s). I don't expect every triager to be a guru at all of these, so my guide incorporates a workflow that should allow triagers with differing skills to work together efficiently.
I'm going to work with the QA team to organize some "crash bug days" on IRC, but I'm interested in feedback before that as well.
June 29th, 2009 at 10:20 pm
Jesse, that page looks good. Thanks for writing it!
One “major†item I have, and this may be an artefact of how I work with Bugzilla, is that I don’t see any mention of moving bugs between components (and at what point in the triage workflow to do so). If you, the triager following the guide, can determine that a crash is a Mac-only font crash, or it’s happening in widget code, or whatever, when do you pass the bug off to that component? Only after you’ve completed the workflow and set the bug to NEW? As soon as you have a good idea of the appropriate component? Some other point? I watch Cocoa Widgets and Mac bugs in Thebes and Plug-ins, so I’m able to apply my platform expertise and knowledge of similar bugs to shepherd the bug on once it arrives in that component, but I’m never in Firefox:General. I’d hate to have triagers spend a lot of time working with many of these sorts of bugs when I could quickly dupe them or ask reporters specific questions to hone in on the source of crash. (On the other hand, I don’t know that this is true of every component, though; I imagine Layout and DOM don’t ever want to see bugs that aren’t NEW+reduced testcase?)
Also, have you checked with QA about the “topcrash†keyword? In the old days I know there were actual, specific criteria for topcrash, and if there still are, the guide shouldn’t be encouraging everyone to add it to the Keywords field “[i]f you discover that there are many reports.â€
Finally, “Don’t be afraid to ask bug reporters to do difficult things, like making a debug build or reducing a testcase, if doing those things on their computer is the only way to figure out the bug†is laudable, but I hope readers of the guide understand that it’s also laughable in many cases ;-) I haven’t triaged Firefox bugs in years so I don’t know the types of reporters you see, but I saw with my own eyes that as Camino became more mainstream, the number of reporters with the ability to do those things declined exponentially to the point where it’s extremely rare. If triagers understand they’re likely to be laughed off most of the time when making those requests, good, but it’s also important that triagers phrase such requests politely and gently when they do make them, so that reporters don’t become irritated or offended (“those @#$@#$ Firefox people expect me to build their program to figure out for them why their program crashes; what jerks!â€) by the request. An irritated reporter with crash you can’t reproduce helps no one, and lots of them harm the goodwill of the project.
All in all, it’s a very logical, focused, and well-written guide!
June 29th, 2009 at 11:56 pm
A crash bug day sounds like something the SUMO community should participate in as well. When you start to organize this event, be sure to drop me a line. Thanks!
July 1st, 2009 at 1:16 am
Nice guide. Is there a reason you’re saying to use the whiteboard markers instead of the keywords that already exist. Seems like having two ways to flag a bug as, say, “needs a regression window”, makes it harder to find such bugs.
The existing keywords: regressionwindow-wanted, testcase-wanted, stackwanted
July 1st, 2009 at 2:29 pm
I’m using whiteboard markers instead of keywords because the keywords aren’t specific enough. Does “stackwanted” mean anyone could post a stack, anyone on Linux could post a stack, or we’re just hoping that a stack for a non-reproducible bug will appear magically? Does “testcase-wanted” mean we need a developer to create a testcase from scratch, we need HTML reduced, or we need to figure out how to reproduce a crash with a local testcase?