Crashtest is Mozilla's newest test suite. It provides a very simple way to make sure a page doesn't cause Gecko to crash (or hang or assert or leak). Since I report a lot of crash and assertion and leak bugs, I've been looking forward to being able to use this test suite. (For some bugs, I had been using "== about:blank" and "!= about:blank" reftests, but most of my bugs weren't getting tests checked in at all.)

I spent most of last week making crashtests for old bugs I had reported and checking them in.

So far, these crashtests have found bug 409150, a crash on Windows that happens on Tinderbox but not dveditz's computer, and bug 408746, a crash on Linux that happens on sayrer's computer but not on Tinderbox. They also found a problem in a patch before it was checked in.

The value of crashtests (and reftests) goes beyond keeping regressions out of nightlies. The test suites can be run under Valgrind's Memcheck, potentially finding subtle memory safety bugs that don't normally show up as crashes. The pages can even be used outside of the normal test framework, for example for finding inconsistent-rendering bugs.

Comments are closed.