Language barriers and bugs
A text-input bug affecting many Arabic speakers was discovered when we were about make Firefox 3.5 release candidates, and we chose to include a fix in Firefox 3.5.1 rather than delay Firefox 3.5.
The zero-day vulnerability in Firefox 3.5 was a frequent crash on the front page of a top-500 Russian site, but we didn't find out about it until after we shipped Firefox 3.5.
The reason we found out about these bugs late isn't a lack of non-English beta users. 1.8% of English Firefox users are beta users, and 1.6% of Russian Firefox users are beta users. Arabic beta use is much higher, at 3.5%, perhaps because Firefox 3 was such a large improvement over Firefox 2 for rendering Arabic text.
Instead, the reason may be that people who don't know English can't report bugs easily. They can't even search Bugzilla to find out whether their bug is known. They went out of their way to try betas, but beta-testing alone does not keep major problems from making their way into a release.
How can we enable the rest of the world to participate in reporting bugs, or at least ensure that we find out about the most frequent problems in other languages?
July 18th, 2009 at 2:26 am
Multilingual Bugzilla? Like Facebook, which shows a different language interface to the user depending on location / preference, there could be something for Bugzilla too.
Though it won’t solve the issue of a English bug report in a Chinese Bugzilla interface for example.
July 18th, 2009 at 3:37 am
Also, don’t underestimate the cultural differences.
Let me exaggerate a bit to get the point across: in the Plone project, we really had to push people in China and Japan to file bugs — it’s not in their culture to “complain”, and they have a very different relationship with authority (perceived or otherwise).
We now have really good bug reports and code coming from Japan and China, but it took a while, and quite a lot of outreach and convincing to get there.
I’m not sure whether this has a similar equivalent in Arabic countries. But don’t chalk it all up to the language barrier. :)
July 18th, 2009 at 3:44 am
As solution, we can imagine some local (e.g. russian, chinese, etc.) “sandbox” websites (such sites may even be located on subdomains of mozilla.org like official bugzilla itself) for different languages where users can report bugs using their *native* language and vote for them. Where some limit of votes for specific bug is reached, some english speaking responsible person adds translated report to “real” (english) bugzilla.
Or even bugzilla itself can just have one more field called “Language”, and rest part of system can be similar to described above.
July 18th, 2009 at 5:18 am
I suggest other languages beta-users should be invited to drop their observations in their own language onto local forums managed by Fx l10n teams. It would be the first level of (easy) bugreport. The second level could be assumed by l10n drivers who could do first triage, check for already existing bugreports and finally submit verified bugs on the general “English” bugzilla.
I am aware that an easier trivial access to report at first level would result in more work on second level, but I think it is worth doing it.
July 18th, 2009 at 8:00 am
Our beta firstrun pages aren’t localized, either. That’s likely a contributing factor. But we also need localized bug triagers.
July 18th, 2009 at 11:11 am
Requiring users to report bugs is going to be a bad way to get information in the first place. You have a tool that sends crash logs, surely the analysis tools to find out that a lot of Russian users are experiences crashes on a certain website can be built.
I mean it’d be great if you can also find a nice way for non-english speaking users to file bug reports, but I think it’d be much better if you didn’t require bug reports to isolate problems. Even English speakers don’t all file enough reports. If you solve this problem, you’ll help solve the multilingual problem too.
July 18th, 2009 at 11:13 am
(Sorry by the way, for the mangled grammar and terribly composed sentences in the last comment. I just got up.)
July 18th, 2009 at 4:45 pm
The Windows 7 team recently published a blog post about this: See the section titled “Understanding feedback from around the world†on http://blogs.msdn.com/e7/archive/2009/07/07/engineering-windows-7-for-a-global-market.aspx. In short, bugs can be submitted in any language, and machine translation is used to get a sense of the bug reports for triage.
July 19th, 2009 at 1:35 pm
I’m hoping that some intl bugs get earlier discovery when we have localized builds on a nightly update channel. That’ll make it more interesting for local geeks to follow our development more closely and might make it more obvious to them to file a bug ’cause it’s less of a “someone else probably caught that one already”.
Regarding getting bugs on file, the machine-translation piece might help us at least for some languages. Making filing bugs more obvious might be good enough, and to encourage users to file bugs in their language. To make them useful for our developers, cloning the bug with a MT of the original report might be a good start.
I’m not sure if we need a localized UI for that, too. I usually think that when people claim they need to get the UI translated to be able to do ABC, the UI is likely too complex for people of that technical skill in English, too. So we might get more bang for the buck making that process more user friendly than trying to get a cumbersome process in many languages.
July 20th, 2009 at 6:53 pm
[…] bugs requires substantial effort from users, and requires both English-language and technical skills, so we prefer not to rely exclusively on user-reported bugs. Luckily, crashes […]
July 20th, 2009 at 11:43 pm
The truth is that for a bug to be noticed, it is not enough to just file it. When one does that, no one notices it, or pays any attention. Especially if the bug is not written in very good English or is about a feature that is not easy to test for English speakers (e.g. Arabic writing).
July 20th, 2009 at 11:49 pm
The Mozilla community in Japan handles it like this:
There’s a Japanese language bugzilla http://bugzilla.mozilla.gr.jp/ where Japanese Mozilla users file bugs in Japanese. Other Japanese users test/confirm the bugs, and Masayuki Nakano (who works on i18n issues primarily) takes the confirmed bugs and moves them over to the main English-language Bugzilla. It works pretty well, but perhaps the negatives would be that it takes time for a bug found in the Japanese community to get filed in the US Bugzilla, and perhaps that this particular process needs to have someone pretty fluent in the codebase (in this case, Masayuki) who can answer questions up front and determine which bugs are relevant to be filed in the English Bugzilla.