At least three times today (that I spotted looking back through the ignored errors), a11y tests on Linux debug have blown up, reporting tens or hundreds of thousands of errors, all starting with 1889 ERROR TEST-UNEXPECTED-FAIL | chrome://mochikit/content/a11y/accessible/test_name_nsRootAcc.xul | Test timed out followed by thousands of [SimpleTest/SimpleTest.js, window.onerror] An error occurred - nsIAccessibleEvent is not defined at chrome://mochikit/content/a11y/accessible/events.js:766 Logs I saw: http://tinderbox.mozilla.org/showlog.cgi?log=Firefox-Unittest/1256785900.1256791549.21285.gz Linux mozilla-central test debug everythingelse on 2009/10/28 20:11:40 http://tinderbox.mozilla.org/showlog.cgi?log=Firefox-Unittest/1256763311.1256769275.28481.gz Linux mozilla-central test debug everythingelse on 2009/10/28 13:55:11 http://tinderbox.mozilla.org/showlog.cgi?log=Firefox-Unittest/1256751737.1256758120.31701.gz Linux mozilla-central test debug everythingelse on 2009/10/28 10:42:17 I haven't seen it before, and the first failure was just a couple of runs after bug 524665 landed, so I suppose (without understanding the code in the least) it could be a result of that, rather than just coincidence.

(In reply to comment #0) > followed by thousands of That behavior is tracked in bug 492956.

this might help

Comment on attachment 409031 [details] [diff] [review] patch [Checkin: Comment 4] r=me. Thanks!

landed on 1.9.3 - http://hg.mozilla.org/mozilla-central/rev/aaaa9779a9a2 hope this will fix it

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox-Unittest/1257765708.1257771744.17357.gz Linux mozilla-central debug test everythingelse on 2009/11/09 03:21:48

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox-Unittest/1257791471.1257797839.4994.gz Linux mozilla-central debug test everythingelse on 2009/11/09 10:31:11

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox-Unittest/1257876663.1257883552.16238.gz Linux mozilla-central debug test everythingelse on 2009/11/10 10:11:03

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox-Unittest/1257886520.1257893272.26917.gz Linux mozilla-central debug test everythingelse on 2009/11/10 12:55:20 Many many a11y failures as described, but unlike comment 0 the first failure is: TEST-UNEXPECTED-FAIL | chrome://mochikit/content/browser/browser/components/places/tests/browser/browser_library_views_liveupdate.js | Timed out

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox-Unittest/1257884834.1257891765.10041.gz#err1 Linux mozilla-central debug test everythingelse on 2009/11/10 12:27:14

Well this is annoying.I'm curious to see if this happens on the 192 branch as well. Ginn this seems to hit linux only, have you seen it on open solaris?

worth a try?

(In reply to comment #11) > Created an attachment (id=411593) [details] > a long shot > > worth a try? Can you describe how this is going to help?

(In reply to comment #10) > Ginn this seems to hit linux only, have you seen it on open solaris? I don't have a box running mochitest-plain or mochitest-a11y all the time. I haven't seen this timeout for my manually runs on Solaris.

Comment on attachment 411593 [details] [diff] [review] a long shot [Checkin: Comment 18] This is worth a shot. I was also thinking whether we could check if the window actually opened, and bail if it didn't within a certain amount of time. But I don't know if toolkit allows us to do that.

(In reply to comment #12) > (In reply to comment #11) > > Created an attachment (id=411593) [details] [details] > > a long shot > > > > worth a try? > > Can you describe how this is going to help? Not sure but I think the focus refactor might have changed eventing for non foreground windows; or a least there might be some strange interaction between gecko eventing and the Linux window manager. Making sure the opened window has is front-most and has focus might help. Like I said, it is a long shot :)

Neil your thoughts on focus events and comment #15?

Comment on attachment 411593 [details] [diff] [review] a long shot [Checkin: Comment 18] ok, at least I don't see anything bad here

Landed on central: http://hg.mozilla.org/mozilla-central/rev/e12ee7ca0834 Note: Adding bug 521233 as a (likely) dependency -- I just discovered this bug now.

(In reply to comment #18) > Landed on central: > http://hg.mozilla.org/mozilla-central/rev/e12ee7ca0834 > This didn't work either.

Not sure if my push caused this but so far no Linux fails, but new to me, Windows fails. http://tinderbox.mozilla.org/showlog.cgi?log=Firefox-Unittest/1257981877.1257984401.28000.gz WINNT 5.2 mozilla-central test everythingelse on 2009/11/11 15:24:37

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox-Unittest/1257986719.1257992406.20502.gz#err0 WINNT 5.2 mozilla-central debug test everythingelse on 2009/11/11 16:45:19

The silly part of me want to try only doing the focus only if (LINUX). Let's see if the instances have truly flipped only to Windows.

If so, then it will tell use more about where to look (OS specific)

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox-Unittest/1257989754.1257993019.27386.gz WINNT 5.2 mozilla-central opt test everythingelse on 2009/11/11 17:35:54

Comment on attachment 411871 [details] [diff] [review] fix added windows failures [Checkin: Comment 28] let's try

Comment on attachment 411871 [details] [diff] [review] fix added windows failures [Checkin: Comment 28] My thoughts exactly. :) Let's land this and see what happens.

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox-Unittest/1258038507.1258044884.29070.gz Linux mozilla-central debug test everythingelse on 2009/11/12 07:08:27 s: moz2-linux-slave01 Looks like comment 28 didn't fix it. :(

Daniel is it less noisy at least?

From a quick glance at the log in comment 29 versus the one in comment 24, it appears to be just as noisy as before.

However, it looks like the changeset in comment 28 was targeted at fixing Windows failures -- and the recent orange (comment 29) was on Linux. (I don't really know the backstory here, so I don't know if that's significant.)

(In reply to comment #31) > From a quick glance at the log in comment 29 versus the one in comment 24, it > appears to be just as noisy as before. Just to be clear, I meant noisy == the frequency of this bug :)

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox-Unittest/1258581730.1258593696.15607.gz Linux mozilla-central debug test everythingelse on 2009/11/18 14:02:10 s: moz2-linux-slave13

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox-Unittest/1259675396.1259682883.24231.gz Linux mozilla-central debug test everythingelse on 2009/12/01 05:49:56 s: moz2-linux-slave02

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox-Unittest/1259716936.1259724448.17969.gz Linux mozilla-central debug test everythingelse on 2009/12/01 17:22:16 s: moz2-linux-slave10

bug 513605 looks like the same underlying issue (or at least same symptom), with a different test timing out.

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1261102092.1261111207.23385.gz Linux mozilla-central debug test everythingelse on 2009/12/17 18:08:12 s: moz2-linux-slave01

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1262034124.1262041846.31771.gz Linux mozilla-central debug test everythingelse on 2009/12/28 13:02:04

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1264184354.1264187294.31064.gz Linux mozilla-central debug test mochitest-other on 2010/01/22 10:19:14 s: moz2-linux-slave15

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1264635959.1264638529.24737.gz Linux mozilla-central debug test mochitest-other on 2010/01/27 15:45:59 s: moz2-linux-slave41

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1265747526.1265751128.17627.gz Linux mozilla-central debug test mochitest-other on 2010/02/09 12:32:06 s: moz2-linux-slave06

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1265759399.1265762175.11113.gz Linux mozilla-central debug test mochitest-other on 2010/02/09 15:49:59 s: moz2-linux-slave06

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1265767077.1265771803.23716.gz Linux mozilla-central debug test mochitest-other on 2010/02/09 17:57:57

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1265828593.1265831653.510.gz Linux mozilla-central debug test mochitest-other on 2010/02/10 11:03:13 s: moz2-linux-slave13

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1265835889.1265838318.12383.gz Linux mozilla-central debug test mochitest-other on 2010/02/10 13:04:49 s: moz2-linux-slave08

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1265839144.1265841969.21024.gz Linux mozilla-central debug test mochitest-other on 2010/02/10 13:59:04 s: moz2-linux-slave09

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1265884122.1265886905.6646.gz Linux mozilla-central debug test mochitest-other on 2010/02/11 02:28:42 s: moz2-linux-slave06

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1265876440.1265879188.14732.gz Linux mozilla-central debug test mochitest-other on 2010/02/11 00:20:40 s: moz2-linux-slave41

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1265902178.1265905849.7173.gz Linux mozilla-central debug test mochitest-other on 2010/02/11 07:29:38 s: moz2-linux-slave11

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1265899813.1265903479.12893.gz Linux mozilla-central debug test mochitest-other on 2010/02/11 06:50:13 s: moz2-linux-slave01

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1265928360.1265933013.26893.gz Linux mozilla-central debug test mochitest-other on 2010/02/11 14:46:00 s: moz2-linux-slave07

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1265933791.1265937772.16930.gz Linux mozilla-central debug test mochitest-other on 2010/02/11 16:16:31 s: moz2-linux-slave09

Comment on attachment 426560 [details] [diff] [review] investigative [Checkin: Comment 63] In absence of a better idea to try, let's try this. I really am not sure why this is failing so often.

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1265957634.1265960179.5947.gz#err0 Linux mozilla-central debug test mochitest-other on 2010/02/11 22:53:54 s: moz2-linux-slave12

investigative: http://hg.mozilla.org/mozilla-central/rev/e24c49718527

(In reply to comment #60) > investigative: http://hg.mozilla.org/mozilla-central/rev/e24c49718527 I can't see any changes.

(In reply to comment #61) > (In reply to comment #60) > > investigative: http://hg.mozilla.org/mozilla-central/rev/e24c49718527 > > I can't see any changes. Wild. Let's try again. investigative: http://hg.mozilla.org/mozilla-central/rev/145556db94fb

Uhm I mean: http://hg.mozilla.org/mozilla-central/rev/4b18d31e83f4 /me grabs coffee

Sigh. Linux mozilla-central debug test mochitest-other http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1265990792.1265993637.31660.gz (Maybe it will get more frequent now that we don't especially focus the dialog on linux)

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1265999005.1266003076.14356.gz Linux mozilla-central debug test mochitest-other on 2010/02/12 10:23:25 s: moz2-linux-slave09

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1266002363.1266005337.8409.gz Linux mozilla-central debug test mochitest-other on 2010/02/12 11:19:23 s: moz2-linux-slave08

We should probably stop the noise and figure this out offline; or take stabs at it during off peak hours.

Comment on attachment 426727 [details] [diff] [review] bail on linux Cancelling review; I have to step away - Alexander can tidy this up and land it.

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1266446100.1266448570.901.gz&fulltext=1 Linux mozilla-central debug test mochitest-other [testfailed] Started 14:35, finished 15:17

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1266483229.1266485625.24659.gz Linux mozilla-central debug test mochitest-other on 2010/02/18 00:53:49 s: moz2-linux-slave08

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1266528454.1266532186.23863.gz Linux mozilla-central debug test mochitest-other on 2010/02/18 13:27:34 s: moz2-linux-slave02

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1266543457.1266546436.13845.gz s: moz2-linux-slave102966 ERROR TEST-UNEXPECTED-FAIL | chrome://mochikit/content/a11y/accessible/test_name_nsRootAcc.xul | Test timed out. 2969 ERROR TEST-UNEXPECTED-FAIL | chrome://mochikit/content/a11y/accessible/test_nsIAccessNode_utils.html | [SimpleTest/SimpleTest.js, window.onerror] An error occurred - uncaught exception: [accessible/events.js, gA11yEventObserver.observe] This is expected if a previous test has been aborted... Initial exception was: [ ReferenceError: nsIAccessibleEvent is not defined ] at :0 PROCESS-CRASH | automation.py | application crashed (minidump found) Thread 1 (crashed) PROCESS-CRASH | automation.py | application crashed (minidump found) Thread 1 (crashed) PROCESS-CRASH | automation.py | application crashed (minidump found) Thread 1 (crashed)

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1266576297.1266578783.30714.gz Linux mozilla-central debug test mochitest-other on 2010/02/19 02:44:57 s: moz2-linux-slave15

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1266601430.1266604090.16626.gz Linux mozilla-central debug test mochitest-other on 2010/02/19 09:43:50 s: moz2-linux-slave02

Comment on attachment 426825 [details] [diff] [review] bail on linux2 [Checkin: Comment 78] Yep, thanks. Pushed as http://hg.mozilla.org/mozilla-central/rev/899cf3b6f797 Sorry for the oranges folks.

Should this still be a random orange bug?

Removed orange keyword and morphed bug summary.

Linux mozilla-central debug test mochitest-other [testfailed] Started 12:02, finished 12:52 http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1266609738.1266612717.13303.gz

(In reply to comment #80) > Removed orange keyword and morphed bug summary. I would not have changed them, as this is worked around, not fixed: yet less need to find this bug if the failure doesn't occur anymore, it may just miss some attention.

this does not look like a browser chrome test, how can it depend on bug 521233?

(In reply to comment #83) > this does not look like a browser chrome test, how can it depend on bug 521233? a11y tests are chrome tests.

(In reply to comment #84) > (In reply to comment #83) > > this does not look like a browser chrome test, how can it depend on bug 521233? > > a11y tests are chrome tests. and a11y tests use waitForFocus function so it might be related, theoretically I think

Right, I set the dependency so I would remember retry this test when 532233 is fixed.

(In reply to comment #84) > (In reply to comment #83) > > this does not look like a browser chrome test, how can it depend on bug 521233? > > a11y tests are chrome tests. that bug is only about BROWSER chrome tests, nobody ever reported issues with waitForFocus in Chrome tests. if you can tell the issue is waitForFocus, than we should change that bug to be generic.

fixed by bug 719754