Indistinguishable from Jesse

I filed my 3000th bug today. It looks like I've passed Gemal, but I'm still trailing Seth and timeless.

Schedule Chicken is a game that nobody wins.

-- Mike Schroepfer, at this morning's Gecko status meeting.

It looks like Robert Sayre is implementing Gerv's Content Restrictions proposal. This will give web sites a way to specify where in the page scripts are allowed to appear. The idea is that if a site using Content Restrictions has an XSS hole (e.g. due to incorrectly escaped text or incorrectly sanitized user HTML), it won't be exploitable in browsers that support Content Restrictions.

I'd rather see anti-XSS effort focused on improving server-side libraries. For example, if web applications were written in terms of "constructing an HTML DOM tree" or "pushing and popping HTML elements", much less code would have to be audited for "remembering to escape output" (I hate that phrase). But it's hard to argue against something that seems like it will help mitigate holes in existing web applications.

Readers who found this interesting might also enjoy Mozilla's metabug for anti-XSS proposals (dependency tree).

Opera has posted a build with fixes for several crashes found by jsfunfuzz. Cool!

Opera community members posted dozens of comments about it, and I replied to several.

Before the presentation:

• ComputerWorld: Mozilla to give away own security testing tools
• InternetNews: Black Hat Cometh; You Afraid?
• Datamation: Will Mozilla's Fuzzer Break The Web?
• eWeek Channel Insider: Mozilla to Release JavaScript Fuzzer for Firefox

After the presentation:

• PC World Staff Blog: Mozilla Releases Hacker Tools
• InternetNews: Mozilla Puts The Fun in Fuzz
• cnet news.com: Mozilla releases browser testing tools
• cybernetnews: Both Opera and Firefox Benefit from Mozilla’s jsfunfuzz
• InfoWorld: Mozilla shares scanning tool, Firefox 3 features
• Mozilla Delivers Security Tools, Previews Firefox 3 At Black Hat
• TechSpot: Mozilla releases browser-abusing tool
• Avencius: Mozilla helps Opera in fixing JavaScript bugs
• Digg: Mozilla Releases Hacker Tools

Fuzz-testing is usually only used to find crashes and assertion failures, but my JavaScript engine fuzzer goes beyond catastrophic failures when it tests the decompiler. It checks the decompiled code for signs of incorrectness in two ways.

First, it checks that the decompiled code compiles without giving syntax errors. This finds fun bugs like bug 346904 where the decompiler screwed up in an understandable way, as well as bugs like bug 351496 where the decompilation is complete nonsense.

Second, it checks that the decompiled code is canonical -- compiling and decompiling again should give the exact same representation as the original decompilation. This helps find bugs like bug 381196 where decompilation changes the meaning of the code without introducing a syntax error.

Some decompilation changes, such as bug 352068, did not change the meaning of the code and simply reflected varying amounts of optimization in the compiler. Early in the fuzzer's life, I was able to convince Brendan that it was worth fixing many of those otherwise harmless "round-trip changes" in order to make it possible to find other bugs with this method.

This pair of checks doesn't find all decompiler bugs, of course, but it finds quite a few of them. jsfunfuzz has a few other correctness checks for things like unnecessary parentheses in decompiled code and bogus results from object uneval.

Can you think of other ways to use fuzz-testing to find "correctness" bugs?

I wrote a fuzzer called jsfunfuzz for testing the JavaScript engine in Firefox. Window, Shaver, and I announced it at Black Hat earlier today, as part of Mozilla's presentation, "Building and Breaking the Browser".

It tests the JavaScript language engine itself, not the DOM. (That means that it works with language features such as functions, objects, operators, and garbage collection rather than DOM objects accessed through "window" or "document".)

It has found about 280 bugs in Firefox's JavaScript engine, over two-thirds of which have already been fixed (go Brendan!). About two dozen were memory safety bugs that we believe were likely to be exploitable to run arbitrary code.

In the presentation, I speculated as why it has been able to find so many bugs:

• It knows the rules of the JavaScript language, allowing it to get decent coverage of combinations of language features.
• It breaks the rules, allowing it to find errors in syntax error handling such as bug 350415 and more generally helping the fuzzer avoid having "blind spots".
• It isn't afraid to nest JavaScript constructs in fairly complicated ways, like when it found bug 353079.
• It allows state to accumulate by creating and running functions in a loop. (See bug 361346 for an example of a bug that would be hard to find otherwise.)
• It tests for correctness, not just crashes and assertions. (Since I didn't talk about this aspect much during the security-focused Black Hat presentation, I've made it a separate blog post.)

If you want to test it out, grab it from the bug report. I recommend running it in a standalone JavaScript Shell, as it is much faster to start and shut down than a whole browser.

Update (2015): newer versions are available in the funfuzz repository on GitHub

This will be my first year at the Black Hat conference in Las Vegas. I'm excited and nervous ;)

Update: I'll be sticking around for DEF CON, too.