Indistinguishable from Reality

User experience designers Alex Faaborg and Alexander Limi are looking to broaden the scope of efforts to make Firefox faster. Until recently, most of the effort has involved reducing the computation time needed to launch Firefox or render a web page. Faaborg and Limi argue that we should also look for ways to make computation time matter less.

The wiki page Perceived Performance contains a long list of ideas. Don't be fooled by the "perceived" part of the name: although a few of the ideas would merely make Firefox users happier, most of the ideas are aimed at reducing the amount of time users spend waiting for Firefox to do various things.

Common themes include:

Improve handling and feel during input. For example, letting scrolling "accelerate" makes rapid scrolling easier without harming the ability to make fine adjustments.

Give instant feedback in response to input, even if an operation has to continue in the background. In addition to reducing human waiting time, these fixes make the experience feel more like direct manipulation and less like telling Firefox what to do. (See also: "snappiness" bugs.)

Allow users to interact with partially loaded pages sooner. This is especially important with slow connections, which are becoming common again as computing goes mobile. Boris Zbarsky's interruptible reflow work is likely to help, and I added some ideas about making better use of the cache.

Reduce the number of clicks and keypresses needed for the most common interactions.

Give users new tools that let them avoid waiting. The introduction of tabbed browsing made it matter less when pages loaded slowly, by letting users load links in background tabs while continuing to read, but also introduced new problems.

Diminish frustration when slowness can't be helped by making changes to activity indicators, progress bars, and other messages. Give users the impression that Firefox is working hard, and help users make better choices about whether to wait.

What kinds of slowness do you encounter while using Firefox? Where should we focus performance efforts, whether by reducing computation time or through more clever means? Can you think of new ways to apply the themes above, or any other ways to make Firefox faster where it matters?

Steps to reproduce:

1. Select "About Mozilla Firefox" from the "Help" menu.
2. Click the "Credits" button.
3. Read the credits as they scroll by.
4. After the end of the credits list (don't bother trying to read the copyright stuff at the bottom), look at somthing else, such as the titlebar of the About Mozilla Firefox dialog.

Result:

At least for me, the titlebar appears to sag.

I discovered arbitrary code execution holes in Firefox, Internet Explorer, and Opera that involve human reaction time. One version of the attack works like this:

The page contains a captcha displaying the word "only" and asks you to type the word to verify that you are a human. As soon as you type 'n', the site attempts to install software, resulting in a security dialog. When you type 'y' at the end of the word, you trigger the 'Yes' button in the dialog. I made a demo of this attack for Firefox and Mozilla.

Another form of the attack involves convincing the user to double-click a certain spot on the screen. This spot happens to be the location where the 'Yes' button will appear. The first click triggers the dialog; the second click lands on the 'Yes' button. I made a demo of this attack for Firefox and Mozilla.

These types of attack work on any security dialog that can be triggered by untrusted content. The attack is most useful in a dialog where one of the buttons means "Yes, let this untrusted content run arbitrary code". Firefox has such a dialog in the form of the extension installation (XPI) dialog. Similarly, Internet Explorer has the ActiveX installation dialog and Opera has an "Open" button for downloaded executables. Programs other than browsers might also be vulnerable.

Firefox's solution, from bug 162020, is to delay enabling the "Yes"/"Install" buttons until three seconds after the dialog appears. I believe that this is the only possible fix other than completely denying untrusted content the ability to pose the dialog. Unfortunately, this fix is frustrating for users who install extensions often.

Some users have been intentionally lowering the delay to 0 seconds, which frustrates me. These users think the delay was added merely to force everyone to read the dialog. It surprises me that these users were not able to figure out the security hole given the fix. Ironically, advanced users are the most susceptible to these attacks, because they type and double-click faster than they react to unexpected stimuli.

It might be possible to lower the delay to less than three seconds, making it less annoying, without jeopardizing security. Designing experiments to determine the minimum "safe" delay would be tricky. You would want to do everything an attacker could do to increase participants' reaction time: give them a complicated task, make new rectangles appear every second to make the dialog less unexpected, etc.

It might make sense to make the dialog appear only after the user clicks a statusbar indicator that means "This web site wants to install software". This would get rid of the problem of choosing a delay, and it wouldn't require users who want to install extensions to wait.

The woman in this clever clothing advertisement is as convincing as the white triangle in the Kanizsa Triangle illusion.

Michelle, Lauren, and I stumbled on a strong illusion last night. It's similar to the checkerboard illusion but involves color rather than just shades of gray.

The "blue" tiles on top of the left cube and the "yellow" tiles on top of the right cube are actually the same shade of gray.

Articles that talk about this illusion: American Scientist: Why We See What We Do and Discover Magazine: Sensory Reflexes. (The authors of the American Scientist article wrote a book with the same name.)

Berkeley's dilemma (as described by the American Scientist article) reminds me of Quine's Gavagai problem in the acquisition of language. Berkeley's dilemma is that retinal images are inherently ambiguous -- for example, there's no difference in the retinal image created by a large object at medium distance and a small object at a large distance. In the Gavagai problem, an island native points to a rabbit and says "gavagai". Do you interpret "gavagai" as "rabbit", "there goes a rabbit", "white", "animal", "hopping", "it's a nice day", "cute", "lunch", or something else?

Both Berkeley's dilemma and the Gavagai problem are problems of infinite ambiguity. Humans have clever heuristics for dealing with both problems. Examples include color constancy and overestimation of acute angles in visual perception, and the whole-object, taxonomic, and mutual-exclusivity assumptions children use to interpret new nouns.

My friend Michelle sees English letters and some Chinese characters as having colors. She doesn't like sites about letter-color synesthesia because they always get the colors of the letters wrong.

Hint added Jan 22:

MIT Perceptual Science Group: lightness; interaction between motion and form

Akiyoshi Kitaoka: static images that appear to move