Better error summaries on Tinderbox

I recently landed a fix so that when Firefox crashes or hangs on Tinderbox, the error summary shows which test was running.

As we add more tests and platforms to Tinderbox, it's increasingly important for developers to be able to identify each test failure quickly and accurately. Good error summaries make assisted starring of random oranges possible, which greatly reduces the pain induced by intermittent failures. Good error summaries also make it possible to track which failures are most frequent, and therefore concentrate on fixing the most important ones.

Error summaries for crashes could be further improved by showing what kind of process crashed and a signature based on the stack trace.

I'd also like to see better error summaries for memory leaks, compiler errors, python errors, and failures in smaller build steps.

If you see other error summaries on Tinderbox that could be improved, please file bugs. It's an easy way to help Mozilla scale across branches, and it's cheaper than cloning philor.

2 Responses to “Better error summaries on Tinderbox”

  1. Justin Dolske Says:

    I nominate you for new Tinderbox module owner. :)

  2. Luke Wagner Says:

    I love you man!