Closed Bug 753225 Opened 12 years ago Closed 11 years ago

Intermittent browser_dbg_createRemote.js | Test timed out | Found a after previous test timed out | leaked 1352 bytes

Categories

(DevTools :: Debugger, defect, P3)

Product:
DevTools
Component:
Debugger
Type:
defect

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: philor, Unassigned)

References

Details

(Keywords: intermittent-failure)

Attachments

(1 obsolete file) 

https://tbpl.mozilla.org/php/getParsedLog.php?id=11586033&tree=Mozilla-Inbound
Rev3 Fedora 12 mozilla-inbound debug test mochitest-other on 2012-05-08 15:49:49 PDT for push 9c8e72215cc9
slave: talos-r3-fed-048

TEST-INFO | chrome://mochitests/content/browser/browser/devtools/debugger/test/browser_dbg_createChrome.js | profile name: _remote-debug_remote-debug
TEST-INFO | chrome://mochitests/content/browser/browser/devtools/debugger/test/browser_dbg_createChrome.js | Console message: [JavaScript Warning: "ReferenceError: reference to undefined property gProcess._dbgProcess.location" {file: "chrome://mochitests/content/browser/browser/devtools/debugger/test/browser_dbg_createChrome.js" line: 33}]
WARNING: NS_ENSURE_TRUE(mMutable) failed: file ../../../../netwerk/base/src/nsSimpleURI.cpp, line 293
INFO TEST-END | chrome://mochitests/content/browser/browser/devtools/debugger/test/browser_dbg_createChrome.js | finished in 477ms
TEST-INFO | checking window state
TEST-START | chrome://mochitests/content/browser/browser/devtools/debugger/test/browser_dbg_createRemote.js
++DOCSHELL 0xdb0b890 == 35 [id = 2380]
++DOMWINDOW == 100 (0xcbdff78) [serial = 6330] [outer = (nil)]
JavaScript error: , line 0: nothing active on context
JavaScript error: , line 0: nothing active on context
++DOMWINDOW == 101 (0xdddba60) [serial = 6331] [outer = 0xcbdff30]
WARNING: NS_ENSURE_TRUE(mMutable) failed: file ../../../../netwerk/base/src/nsSimpleURI.cpp, line 293
WARNING: NS_ENSURE_TRUE(mMutable) failed: file ../../../../netwerk/base/src/nsSimpleURI.cpp, line 293
WARNING: No outer window available!: file ../../../dom/base/nsGlobalWindow.cpp, line 8434
WARNING: No outer window available!: file ../../../dom/base/nsGlobalWindow.cpp, line 8434
WARNING: failed to bind socket: file ../../../../netwerk/base/src/nsServerSocket.cpp, line 313
TEST-UNEXPECTED-FAIL | chrome://mochitests/content/browser/browser/devtools/debugger/test/browser_dbg_createRemote.js | an unexpected uncaught JS exception reported through window.onerror - uncaught exception: 2147746065 at :0
Stack trace:
    JS frame :: chrome://mochikit/content/tests/SimpleTest/SimpleTest.js :: simpletestOnerror :: line 983
    native frame :: <unknown filename> :: <TOP_LEVEL> :: line 0
    native frame :: <unknown filename> :: <TOP_LEVEL> :: line 0

JavaScript error: , line 0: uncaught exception: 2147746065
TEST-INFO | chrome://mochitests/content/browser/browser/devtools/debugger/test/browser_dbg_createRemote.js | Console message: [JavaScript Error: "uncaught exception: 2147746065"]
...
TEST-UNEXPECTED-FAIL | chrome://mochitests/content/browser/browser/devtools/debugger/test/browser_dbg_createRemote.js | Test timed out
(screenshot)
TEST-UNEXPECTED-FAIL | chrome://mochitests/content/browser/browser/devtools/debugger/test/browser_dbg_createRemote.js | Cleanup function threw an exception at chrome://browser/content/tabbrowser.xml:1511 - TypeError: aTab is null
TEST-UNEXPECTED-FAIL | chrome://mochitests/content/browser/browser/devtools/debugger/test/browser_dbg_createRemote.js | leaked window property: _remoteDebugger
INFO TEST-END | chrome://mochitests/content/browser/browser/devtools/debugger/test/browser_dbg_createRemote.js | finished in 30178ms
TEST-UNEXPECTED-FAIL | chrome://mochitests/content/browser/browser/devtools/debugger/test/browser_dbg_createRemote.js | Found a tab after previous test timed out: http://example.com/browser/browser/devtools/debugger/test/browser_dbg_iframes.html
WARNING: NS_ENSURE_TRUE(mMutable) failed: file ../../../../netwerk/base/src/nsSimpleURI.cpp, line 293
--DOCSHELL 0xe7d9df8 == 12 [id = 2382]
TEST-INFO | checking window state
TEST-UNEXPECTED-FAIL | chrome://mochitests/content/browser/browser/devtools/debugger/test/browser_dbg_createRemote.js | Found a  after previous test timed out
TEST-UNEXPECTED-FAIL | chrome://mochitests/content/browser/browser/devtools/debugger/test/browser_dbg_createRemote.js | Found a  after previous test timed out
I believe this is the root cause:

WARNING: failed to bind socket: file ../../../../netwerk/base/src/nsServerSocket.cpp, line 313

One way to avoid it is to change the pref (up to 2-3 times) to use different port numbers if the default fails.
I can take this.
Assignee: nobody → vporof
Status: NEW → ASSIGNED
Priority: -- → P3
Attached patch v1 (obsolete) (deleted) — — Splinter Review 
Going through try.
(In reply to Victor Porof from comment #44)
> Created attachment 629735 [details] [diff] [review]
> v1
> 
> Going through try.

All green, but testing a bit more: https://tbpl.mozilla.org/?tree=Try&rev=286c85a4a745
Comment on attachment 629735 [details] [diff] [review]
v1

Review of attachment 629735 [details] [diff] [review]:
-----------------------------------------------------------------

LGTM.
Attachment #629735 - Flags: review+
Whiteboard: [orange] → [orange][land-in-fx-team]
https://hg.mozilla.org/integration/fx-team/rev/6f0a331cf5ec
Whiteboard: [orange][land-in-fx-team] → [orange][fixed-in-fx-team]
https://hg.mozilla.org/mozilla-central/rev/6f0a331cf5ec
Status: ASSIGNED → RESOLVED
Closed: 12 years ago
Resolution: --- → FIXED
Whiteboard: [orange][fixed-in-fx-team] → [orange]
Target Milestone: --- → Firefox 16
Comment on attachment 629735 [details] [diff] [review]
v1

[Approval Request Comment]
Bug caused by (feature/regressing bug #): new feature
User impact if declined: test fix only. No impact.
Testing completed (on m-c, etc.): on m-c
Risk to taking this patch (and alternatives if risky): tests might not turn orange.
String or UUID changes made by this patch:
Attachment #629735 - Flags: approval-mozilla-aurora?
Attachment #629735 - Flags: approval-mozilla-aurora? → approval-mozilla-aurora+
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Will talk about this next triage.
Assignee: vporof → nobody
Summary: Intermittent browser_dbg_createRemote.js | an unexpected uncaught JS exception reported through window.onerror - uncaught exception: 2147746065 at :0, Test timed out, Cleanup function threw an exception, Found a tab... → Intermittent browser_dbg_createRemote.js | Test timed out | Found a after previous test timed out
This started being a bit too chatty during the last couple of days, and I'm quite convinced bug 787181 isn't the cause.

Here's a short excerpt of what's happening, edited for brevity:

>> TEST-START
>> TEST-INFO | Attempting to open a new listener on port 6001
Not using the default port here by design.
>> TEST-INFO | Closing listener...
>> TEST-INFO | Opening listener...
>> TEST-INFO | Console message: [JavaScript Error: "Couldn't connect to localhost:6001" {file: "chrome://browser/content/debugger-controller.js" line: 147}]
Actually, the screenshot tells a different story. It shows the remote debugger up and running paused on the debugger statement as expected.
>> TEST-INFO | Current remote port: 6001
By this time, the connection has succeeded, since we're after a 'resumed' event.
>> TEST-INFO | Current autoconnect flag: true
>> TEST-PASS | The close button should be hidden in a remote debugger.
>> TEST-PASS | Should be running after debug_remote.
>> TEST-PASS | Found the iframe
>> TEST-INFO | Console message: [JavaScript Error: "Error: fillFrames command sent while not paused.: TC_assertPaused@resource://gre/modules/devtools/dbg-client.jsm:501
And boom, after a runDebuggerStatement call.
>> TEST-UNEXPECTED-FAIL | Test timed out
:(
Summary: Intermittent browser_dbg_createRemote.js | Test timed out | Found a after previous test timed out → Intermittent browser_dbg_createRemote.js | Test timed out | Found a after previous test timed out | leaked 1352 bytes
(In reply to Victor Porof [:vp] from comment #88)
> This started being a bit too chatty during the last couple of days, and I'm
> quite convinced bug 787181 isn't the cause.
> 
> Here's a short excerpt of what's happening, edited for brevity:
> 
> >> TEST-START
> >> TEST-INFO | Attempting to open a new listener on port 6001
> Not using the default port here by design.
> >> TEST-INFO | Closing listener...
> >> TEST-INFO | Opening listener...
> >> TEST-INFO | Console message: [JavaScript Error: "Couldn't connect to localhost:6001" {file: "chrome://browser/content/debugger-controller.js" line: 147}]
> Actually, the screenshot tells a different story. It shows the remote
> debugger up and running paused on the debugger statement as expected.
> >> TEST-INFO | Current remote port: 6001
> By this time, the connection has succeeded, since we're after a 'resumed'
> event.
> >> TEST-INFO | Current autoconnect flag: true
> >> TEST-PASS | The close button should be hidden in a remote debugger.
> >> TEST-PASS | Should be running after debug_remote.
> >> TEST-PASS | Found the iframe
> >> TEST-INFO | Console message: [JavaScript Error: "Error: fillFrames command sent while not paused.: TC_assertPaused@resource://gre/modules/devtools/dbg-client.jsm:501
> And boom, after a runDebuggerStatement call.
> >> TEST-UNEXPECTED-FAIL | Test timed out
> :(

More importantly, right before the fillFrames error, an out of memory condition is triggered. My guess is that the engine tries to do a GC and it doesn't like the fact that the window is paused from hitting the debugger statement.

CCing some people who may know more.
past, do you have any more ideas for this one? It is one of our top oranges at the moment...
I've been looking at this with jimb and he pointed out that in the cases we looked at the stack was busted, so I think this increases the likelihood that this is an OOM situation that we don't handle as we should. I wish someone like billm could take a look at it.
Top-orange that makes sheriffing harder and causes the catch-22 of too many failures so hard to spot new intermittents quickly enough as they creep in/when bisecting.

Disabled on Windows for now (with Makefile warning so it doesn't get forgotten):
https://hg.mozilla.org/integration/mozilla-inbound/rev/763cf0e2df5e
Whiteboard: [orange] → [orange][test disabled on Windows][leave open]
(In reply to Ed Morley [:edmorley UTC+1] from comment #217)
> Disabled on Windows for now (with Makefile warning so it doesn't get
> forgotten):
> https://hg.mozilla.org/integration/mozilla-inbound/rev/763cf0e2df5e

https://hg.mozilla.org/releases/mozilla-beta/rev/f512f07621f9
status-firefox15: fixed → ---
Target Milestone: Firefox 16 → ---
My OOM workaround theory didn't pan out:
https://tbpl.mozilla.org/?tree=Try&rev=ab3b0cf2e623

I need to talk to someone who understands this assertion.
OS: Linux → All
Hardware: x86 → All
I think there may be some connection between this and bug 774011.
Look at the full log, before the timeout:

TEST-INFO | Console message: [JavaScript Error: "Error: fillFrames command sent while not paused.: TC_assertPaused@resource://gre/modules/devtools/dbg-client.jsm:585
TC_fillFrames@resource://gre/modules/devtools/dbg-client.jsm:913
SF__onPaused@chrome://browser/content/debugger-controller.js:388
EV_notify@resource://gre/modules/devtools/dbg-client.jsm:144
TC_onThreadState@resource://gre/modules/devtools/dbg-client.jsm:1041
DC_onPacket@resource://gre/modules/devtools/dbg-client.jsm:478
@chrome://global/content/devtools/dbg-transport.js:169
" {file: "resource://gre/modules/devtools/dbg-client.jsm" line: 148}]

vs.

TEST-UNEXPECTED-FAIL | Should be paused after an interrupt request. - Got false, expected true
TEST-UNEXPECTED-FAIL | Should have one frame in the stack. - Got 0, expected 1

Somehow, after framesadded the debuggee isn't in a paused state anymore.
Toporange with several hundred failures not telling us anything new. Was already disabled on Windows, now disabled on the other platforms as well, as part of the drive to reduce our OrangeFactor:
https://hg.mozilla.org/integration/mozilla-inbound/rev/4a5e292d6b0d
Whiteboard: [orange][test disabled on Windows][leave open] → [orange][test disabled][leave open]
Whiteboard: [orange][test disabled][leave open] → [test disabled][leave open]
Attachment #629735 - Attachment is obsolete: true
Depends on: 823577
Test removed in bug 823577.
Status: REOPENED → RESOLVED
Closed: 12 years ago → 11 years ago
Resolution: --- → FIXED
Whiteboard: [test disabled][leave open]
Product: Firefox → DevTools
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: