My DOM fuzzer can now find bugs where the layout of a DOM tree depends on its history.
In this example, forcing a re-layout swapped a “1” and “3” on the screen. My fuzzer didn’t know which rendering was correct, but it could tell that Firefox was being inconsistent.
|Initial DOM tree||
remove the inner span
Gecko developer Simon Montagu quickly determined that 13ت is the correct rendering and attached a patch. Later, when a user reported that the bug affected Persian comments on Facebook, we were able to backport Simon’s fix to Firefox 11.
How it works
The fuzzer starts by making random dynamic changes to a page. Then it compares two snapshots: one taken immediately after the dynamic changes, and another taken after also forcing a relayout.
To force a relayout, it removes the root from the document and then adds it back:
var r = document.documentElement; document.removeChild(r); document.appendChild(r);
In theory, I could also look for bugs where dynamic changes do not repaint enough of the window. But I've been told that testing for painting invalidation bugs is tricky, so I'll wait until most of the layout bugs are fixed.
Since the testcases are random, I have to be heavy-handed in ignoring known bugs. If I file a rendering bug where the weirdest part of the testcase is floats, I'll have the fuzzer ignore inconsistent rendering in testcases with floats until the bug is fixed.
The current list of exceptions is fairly large and includes key web technologies:
- CSS border/padding (bug 718452)
- CSS position: relative/absolute (bug 728100)
- CSS float (bug 725928)
- Non-ASCII text (bug 726460)
- Right-to-left text (bug 730562)
- <table> (bug 467723)
- MathML (bug 522393)
- SVG (bug 723376, bug 475216)
- Anything that causes coordinate overflow
- Anything that causes assertion failures (which are tracked separately)