Closed Bug 1517210 Opened 6 years ago Closed 6 years ago

Remove the {freshCompartment: true} option from the devtools sandbox

Categories

(DevTools :: Debugger, enhancement, P3)

Product:
DevTools
Component:
Debugger
Type:
enhancement

Tracking

(firefox67 fixed)

Status:
RESOLVED FIXED
Milestone:
Firefox 67
Tracking Flags:
Tracking Status
firefox67 --- fixed

People

(Reporter: jandem, Assigned: ochameau)

References

(Depends on 2 open bugs, Blocks 2 open bugs)

Details

Attachments

(1 file, 1 obsolete file)

Bug 1517210 - Stop always loading DevTools modules in fresh compartments.
(deleted), text/x-phabricator-request
Details 
I'll add this flag in bug 1514210 or bug 1512029 so I'm not blocked on devtools test fixes happening in bug 1515290, but we should remove it after that work is done to improve performance significantly.
Priority: -- → P3

I finally got a green try for this and results look very promising:
https://treeherder.mozilla.org/perf.html#/comparesubtest?originalProject=try&originalRevision=1b9abddc44aff2833fa27d1b667615c1660c6b6f&newProject=try&newRevision=015c2c127b8b7cf8fd33f80618ceadabb64b254d&originalSignature=1759151&newSignature=1759151&framework=12
I've not verified if all the cross compartments are gone, but we get very significant win across DevTools.

There is still all the dependents bugs to be landed before being able to get to this one.

Assignee: nobody → poirot.alex

(In reply to Alexandre Poirot [:ochameau] from comment #1)

I finally got a green try for this a results look very promising:

Great results! \o/

Now, DevTools server is loaded with a custom loader every time we want to debug chrome
resources. We ensure toggle the "invisibleToDebugger" flag on Loader.jsm which itself propagates to DevTools Sandboxes.

Pushed by apoirot@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/ae859b463698
Stop always loading DevTools modules in fresh compartments. r=jandem

Unfortunately, this patch raises an intermittent/almost permafailure on a couple of jsterm test, that, only on Windows 7!

https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=225419083&repo=try&lineNumber=2137

The most reproducible failure is this one, failing when we re-run this test with codemirror turned on.
Unfortunately, I was unable to reproduce it locally on linux. Same as all other tests, it seems to be a failure during toolbox opening.
Compare to the two other failure, this one may fail during the call to showToolbox, done right after the logging of "Opening the toolbox" done over here:
https://searchfox.org/mozilla-central/source/devtools/client/shared/test/shared-head.js#405
The other tests seem to fail after openToolboxForTab resolves completely.

10:42:50     INFO - TEST-PASS | devtools/client/webconsole/test/mochitest/browser_jsterm_completion.js | non-object completion - 
10:42:50     INFO - Expects "           ice", is "           ice"
10:42:50     INFO - TEST-PASS | devtools/client/webconsole/test/mochitest/browser_jsterm_completion.js | string literal completion - 
10:42:50     INFO - Adding a new tab with URL: data:text/html;charset=utf8,<p>test code completion
10:42:50     INFO -   <script>
10:42:50     INFO -     foobar = true;
10:42:50     INFO -   </script>
10:42:50     INFO - Tab added and finished loading
10:42:50     INFO - Opening the toolbox
10:42:50     INFO - Buffered messages finished
10:42:50     INFO - TEST-UNEXPECTED-FAIL | devtools/client/webconsole/test/mochitest/browser_jsterm_completion.js | Test timed out -

https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=225419085&repo=try&lineNumber=2396

This one fails with an OOM when opening the toolbox with codemirror turned on.

10:46:32     INFO - TEST-PASS | devtools/client/webconsole/test/mochitest/browser_jsterm_history.js | check history next idx:2 - 
10:46:32     INFO - Adding a new tab with URL: data:text/html;charset=UTF-8,test
10:46:32     INFO - Tab added and finished loading
10:46:32     INFO - Opening the toolbox
10:46:32     INFO - Toolbox opened and focused
10:46:32     INFO - Console message: [JavaScript Error: "uncaught exception: out of memory"]
10:46:32     INFO - Console message: [JavaScript Error: "uncaught exception: out of memory"]
10:46:32     INFO - Console message: [JavaScript Error: "uncaught exception: out of memory"]
10:46:32     INFO - Console message: [JavaScript Error: "uncaught exception: out of memory"]
10:46:32     INFO - Console message: [JavaScript Error: "uncaught exception: out of memory"]
10:46:32     INFO - Console message: [JavaScript Error: "uncaught exception: out of memory"]
10:46:32     INFO - Console message: [JavaScript Error: "uncaught exception: out of memory"]
10:46:32     INFO - Console message: [JavaScript Error: "uncaught exception: out of memory"]
10:46:32     INFO - Console message: [JavaScript Error: "uncaught exception: out of memory"]
10:46:32     INFO - Buffered messages finished
10:46:32     INFO - TEST-UNEXPECTED-FAIL | devtools/client/webconsole/test/mochitest/browser_jsterm_history.js | Test timed out -

https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=225419073&repo=try&lineNumber=2411

This one only times out when re-opening the toolbox with codemirror turned one.

10:44:49     INFO - TEST-PASS | devtools/client/webconsole/test/mochitest/browser_jsterm_helper_pprint.js | `pprint(function)` shows function source - 
10:44:49     INFO - Adding a new tab with URL: data:text/html,Test <code>pprint()</code> jsterm helper
10:44:49     INFO - Tab added and finished loading
10:44:49     INFO - Opening the toolbox
10:44:49     INFO - Console message: [JavaScript Error: "The character encoding of the HTML document was not declared. The document will render with garbled text in some browser configurations if the document contains characters from outside the US-ASCII range. The character encoding of the page must be declared in the document or in the transfer protocol." {file: "data:text/html,Test <code>pprint()</code> jsterm helper" line: 0}]
10:44:49     INFO - Toolbox opened and focused
10:44:49     INFO - Buffered messages finished
10:44:49     INFO - TEST-UNEXPECTED-FAIL | devtools/client/webconsole/test/mochitest/browser_jsterm_helper_pprint.js | Uncaught exception - at chrome://mochitests/content/browser/devtools/client/webconsole/test/mochitest/head.js:78 - TypeError: hud is null
10:44:49     INFO - Stack trace:
10:44:49     INFO - openNewTabAndConsole@chrome://mochitests/content/browser/devtools/client/webconsole/test/mochitest/head.js:78:5
Flags: needinfo?(poirot.alex)

I think there is two distinct issues:

  • something with components.stack versus Cu.callFunctionWithAsyncStack being in distinct compartment for no obvious reason as I dropped the freshCompartment and the failing test, I think, shouldn't be setting invisibleToDebugger flag.
    This exception happens on this try push, with only the patch attached on this bug:
10:42:13     INFO - GECKO(3548) | Handler function executor threw an exception: Error: Must use a native JavaScript stack frame
10:42:13     INFO - GECKO(3548) | Stack: callFunctionWithAsyncStack@resource://devtools/shared/platform/stack.js:61:10

And for some reason, this exception only happens on Windows 7?!!

We do retrieve the stack here and store it here before passing it to callFunctionWithAsyncStack here.

      const stack = getStack();
      executor = () => {
        callFunctionWithAsyncStack(fn, stack, "DevToolsUtils.executeSoon");
      };
  • OOM crashes.
    I thought that the first issue about stacks was the root issue behind toolbox failing to open, so I tried to address it via this patch, but I'm now only having OOM crashes on this try push.
    Now, this workaround patch may be wrong. It still let callFunctionWihtAsyncStack throw. But may be, in some cases, it crashes or introduces crashes later on...

https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=227673383&repo=try&lineNumber=3392

17:13:27     INFO - TEST-START | devtools/client/webconsole/test/mochitest/browser_webconsole_visibility_messages.js
17:13:27     INFO - GECKO(5688) | [GFX1-]: Failed buffer for 0, 0, 1264, 768
...
17:13:28     INFO - GECKO(5688) | console.log: "[DISPATCH] action type:" "UPDATE_GRIDS"
17:13:28     INFO - GECKO(5688) | console.log: "[DISPATCH] action type:" "CLEAR_FLEXBOX"
17:13:28     INFO - GECKO(5688) | console.log: "[DISPATCH] action type:" "UPDATE_OFFSET_PARENT"
17:13:28     INFO - GECKO(5688) | console.log: "[DISPATCH] action type:" "UPDATE_LAYOUT"
17:13:28     INFO - GECKO(5688) | console.log: "[DISPATCH] action type:" "CLEAR_FLEXBOX"
17:13:28     INFO - GECKO(5688) | console.log: "[DISPATCH] action type:" "UPDATE_OFFSET_PARENT"
17:13:28     INFO - GECKO(5688) | console.log: "[DISPATCH] action type:" "UPDATE_LAYOUT"
17:13:28     INFO - GECKO(5688) | [GFX1-]: Failed buffer for 0, 0, 1264, 1016
...
17:13:28     INFO - GECKO(5688) | Assertion failure: [unhandlable oom] Failed not allocate new chunk during GC, at z:/build/build/src/js/src/vm/JSContext.cpp:1486

https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=227673385&repo=try&lineNumber=3431

17:13:13     INFO - TEST-PASS | devtools/client/webconsole/test/mochitest/browser_webconsole_visibility_messages.js | All the messages logged when the console was hidden were displayed. - 
17:13:13     INFO - Leaving test bound 
17:13:13     INFO - Entering test bound 
17:13:13     INFO - Adding a new tab with URL: data:text/html;charset=utf-8,%0A%20%20%3C!DOCTYPE%20html%3E%0A%20%20%3Chtml%3E%0A%20%20%20%20%3Cbody%3E%0A%20%20%20%20%20%20%3Ch1%3ETest%20console%20visibility%20update%3C/h1%3E%0A%20%20%20%20%20%20%3Cscript%3E%0A%20%20%20%20%20%20%20%20function%20log(str)%20%7B%0A%20%20%20%20%20%20%20%20%20%20console.log(str);%0A%20%20%20%20%20%20%20%20%7D%0A%20%20%20%20%20%20%3C/script%3E%0A%20%20%20%20%3C/body%3E%0A%20%20%3C/html%3E%0A
17:13:13     INFO - Tab added and finished loading
17:13:13     INFO - Opening the toolbox
17:13:13     INFO - Buffered messages finished
17:13:13    ERROR - TEST-UNEXPECTED-FAIL | devtools/client/webconsole/test/mochitest/browser_webconsole_visibility_messages.js | application terminated with exit code 1
...
17:13:24     INFO - PROCESS-CRASH | devtools/client/webconsole/test/mochitest/browser_webconsole_visibility_messages.js | application crashed [@ mozalloc_abort]

https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=227673389&repo=try&lineNumber=3168

17:12:39     INFO - TEST-PASS | devtools/client/webconsole/test/mochitest/browser_webconsole_sidebar_scroll.js | Sidebar overflows - 
17:12:39     INFO - Leaving test bound 
17:12:39     INFO - Buffered messages finished
17:12:39    ERROR - TEST-UNEXPECTED-FAIL | devtools/client/webconsole/test/mochitest/browser_webconsole_sidebar_scroll.js | application terminated with exit code 1
...
17:12:49     INFO - PROCESS-CRASH | devtools/client/webconsole/test/mochitest/browser_webconsole_sidebar_scroll.js | application crashed [@ mozalloc_abort]

https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=227673390&repo=try&lineNumber=3419

7:13:22     INFO - TEST-START | devtools/client/webconsole/test/mochitest/browser_webconsole_warn_about_replaced_api.js
17:13:22     INFO - GECKO(6024) | [GFX1-]: Failed buffer for 0, 0, 1264, 768
17:13:22     INFO - GECKO(6024) | [GFX1-]: Failed buffer for 0, 0, 1264, 768
17:13:22     INFO - GECKO(6024) | [GFX1-]: Failed buffer for 0, 0, 1264, 768
17:13:22     INFO - GECKO(6024) | [GFX1-]: Failed buffer for 0, 0, 1264, 768
17:13:22     INFO - GECKO(6024) | [GFX1-]: Failed buffer for 0, 0, 1264, 768
17:13:22     INFO - GECKO(6024) | [GFX1-]: Failed buffer for 0, 0, 1264, 768
17:13:22     INFO - GECKO(6024) | [GFX1-]: Failed buffer for 0, 0, 1264, 768
17:13:22     INFO - GECKO(6024) | [GFX1-]: Failed buffer for 0, 0, 1264, 768
17:13:22     INFO - GECKO(6024) | out of memory: 0x0000000000123D48 bytes requested
17:13:23     INFO - GECKO(6024) | Exiting due to channel error.
17:13:23     INFO - GECKO(6024) | Exiting due to channel error.
17:13:23     INFO - GECKO(6024) | Exiting due to channel error.
17:13:23     INFO - TEST-INFO | Main app process: exit 1
17:13:23     INFO - Buffered messages logged at 17:13:22
17:13:23     INFO - Entering test bound 
17:13:23     INFO - Adding a new tab with URL: data:text/html;charset=utf8,<script>console.log('foo')</script>
17:13:23     INFO - Tab added and finished loading
17:13:23     INFO - Opening the toolbox
17:13:23     INFO - Buffered messages finished
17:13:23    ERROR - TEST-UNEXPECTED-FAIL | devtools/client/webconsole/test/mochitest/browser_webconsole_warn_about_replaced_api.js | application terminated with exit code 1
...
17:13:38     INFO - PROCESS-CRASH | devtools/client/webconsole/test/mochitest/browser_webconsole_warn_about_replaced_api.js | application crashed [@ mozalloc_abort]

Jan, Do you have any idea what could be wrong here? I think there is something around Components.stack and Cu.callFunctionWithAsyncStack, but that can be many other things...

Flags: needinfo?(jdemooij)

It's possible the Components.stack issue is also OOM-related somehow and it manifests like this. Some GC-related changes landed recently so I pushed this on top of inbound tip to see if this still happens:

https://treeherder.mozilla.org/#/jobs?repo=try&group_state=expanded&revision=ee3b6cd710f7aa629e53ae47aa54675a2d39bffe

If it still happens the next step is to force some GCs in this code so we can see if that helps.

This still happens in the web console tests (dt7 now instead of dt4).

I looked at one of the logs. Before we crash we have this in the log:

12:01:55     INFO - GECKO(6132) | MEMORY STAT | vsize 1904MB | vsizeMaxContiguous 4MB | residentFast 1012MB | heapAllocated 404MB
12:01:55     INFO - TEST-OK | devtools/client/webconsole/test/mochitest/browser_webconsole_stacktrace_location_debugger_link.js | took 4227ms

Whereas on inbound I see this:

12:26:31     INFO - GECKO(4180) | MEMORY STAT | vsize 1715MB | vsizeMaxContiguous 124MB | residentFast 699MB | heapAllocated 184MB
12:26:31     INFO - TEST-OK | devtools/client/webconsole/test/mochitest/browser_webconsole_stacktrace_location_debugger_link.js | took 2403ms

Looks like we're running out of virtual address space.

I'll see what happens when we force a shrinking GC between each test...

(In reply to Jan de Mooij [:jandem] from comment #10)

I'll see what happens when we force a shrinking GC between each test...

OK so with this everything is green and the numbers for after the test in comment 10 are better than on inbound:

13:43:54     INFO - GECKO(3528) | MEMORY STAT | vsize 1068MB | vsizeMaxContiguous 314MB | residentFast 464MB | heapAllocated 168MB
13:43:54     INFO - TEST-OK | devtools/client/webconsole/test/mochitest/browser_webconsole_stacktrace_location_debugger_link.js | took 2292ms

So it seems there's not a JS memory leak but somehow we're not GC'ing properly. I'll try to compare GC behavior with inbound.

(In reply to Jan de Mooij [:jandem] from comment #11)

I'll try to compare GC behavior with inbound.

When I dump GC stats to stderr it's green. Maybe it's timing related? I doubt the output would be all that helpful though considering multiple processes are involved.

Andrew, do you have any thoughts on what to try or do next? Win7 dt7 is intermittently OOM'ing with Alexandre's patch (running out of virtual address space), however triggering (shrinking) GCs between tests fixes it...

Flags: needinfo?(jdemooij) → needinfo?(continuation)

(In reply to Jan de Mooij [:jandem] from comment #12)

Andrew, do you have any thoughts on what to try or do next? Win7 dt7 is intermittently OOM'ing with Alexandre's patch (running out of virtual address space), however triggering (shrinking) GCs between tests fixes it...

I think a GC person needs to take a look at this. Presumably creating and destroying a ton of compartments was triggering some kind of GC heuristic to trigger a shrinking GC, and now we're not creating as many compartments. It is a little surprising that your patch didn't help here. Either the heuristics could be tweaked to deal with this better, or you could just force shrinking GCs. Tests in general don't have very reasonable allocation behavior.

Flags: needinfo?(continuation) → needinfo?(pbone)
Depends on: 1528667

I can't read the crash reports, there's no symbols in the stacks and not enough data to work from. See Bug 1528668.

Depends on: 1528668
Flags: needinfo?(pbone)

Alexandre, for now maybe just add Cu.forceShrinkingGC() calls to some of the console tests where we see this. It's not ideal but it unblocks your performance work here.

Flags: needinfo?(poirot.alex)

Yes it seems to make try green:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=65a136a281787bf54c28166a7d6245e01be63467

But I'm concerned about making the console OOM on Windows 7. I is not clear what special the test is doing to introduce such OOM crash.

Flags: needinfo?(poirot.alex)

(In reply to Alexandre Poirot [:ochameau] from comment #16)

But I'm concerned about making the console OOM on Windows 7. I is not clear what special the test is doing to introduce such OOM crash.

I think it has to do with GC heuristics, maybe related to test timing or us allocating a lot less CCWs now. It's unlikely to affect real-world code, as Andrew said in comment 13:

Tests in general don't have very reasonable allocation behavior.

The other option is to wait for the GC team to dig in, but I don't know if/when that will happen (I don't have cycles for it atm).

(In reply to Jan de Mooij [:jandem] from comment #17)

Tests in general don't have very reasonable allocation behavior.

The other option is to wait for the GC team to dig in, but I don't know if/when that will happen (I don't have cycles for it atm).

I'm curious about what is going on here, but a stack trace would be a huge clue, I'd also like to see it in the profiler/with logging but I'd have to setup a windows VM. It'd take more time than I have easily right now and the shrinking GCs might just be better for now. maybe leave the bug open / open a new bug for some investigation / remove the shrinking GCs later.

Multi platform try run:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=cda962c9022b29e8aec676b7c7615f8ff8165af0

Given Paul and Jan's feedback, I'll proceed with the forced GC in console tests and open a followup.

Blocks: 1528986

Now, DevTools server is loaded with a custom loader every time we want to debug chrome
resources. We ensure toggling the "invisibleToDebugger" flag on Loader.jsm which itself
propagates to DevTools Sandboxes.

We added calls to force GC in console test to prevent crashes on Windows 7.

Pushed by apoirot@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/67fac975c2f8
Stop always loading DevTools modules in fresh compartments. r=jandem,nchevobbe

This permafailure looks like the existing intermittent bug 1517936, which has been duplicated with bug 1517817.
These intermittents also throws the Must use a native JavaScript stack frame exception when using Cu.callFunctionWithAsyncStack.
I tried some GC at the beginning and end of the failing test, but it still fails:
https://treeherder.mozilla.org/#/jobs?repo=try&selectedJob=229424396&revision=72ecea68a7536c9f63d9f9bb8f7909a8c74baef9

I just pushed to try another patch, trying to workaround the Cu.callFuntionWithAsyncStack exception (without GCs):
https://treeherder.mozilla.org/#/jobs?repo=try&selectedJob=229424396&revision=3c42b118cfd480923318986fa4190540cd993f6a

I'm also running out of clues for this failure. It doesn't crash with OOM, but the test (devtools/client/inspector/rules/test/browser_rules_user-agent-styles.js) is failing with timeouts at different location in the test with DevTools requests never getting any response (at least it looks like it).

The GC size after the test failure looks very high:

 GECKO(5004) | MEMORY STAT | vsize 1753MB | vsizeMaxContiguous 67MB | residentFast 844MB | heapAllocated 335MB

Does it still means there is an issue with the GC? If yes, why would it still fail with the forced GC in the first try run:
https://hg.mozilla.org/try/rev/d417e70df43c687405ff149ef049a33dbe04b6d3

Flags: needinfo?(poirot.alex)

It looks like we get back to the Components.stack+Cu.callFuntionWithAsyncStack issue.

With additional GCs at the beginning and the end of the failing test (devtools/client/inspector/rules/test/browser_rules_user-agent-styles.js), I still get intermittent failures:
https://treeherder.mozilla.org/#/jobs?repo=try&author=apoirot%40mozilla.com&selectedJob=229449238

But if I workaround the exception of stack, it passes:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=3c42b118cfd480923318986fa4190540cd993f6a
I'm not sure callFunctionWithAsyncStack fails because of memory?

I'm not sure that this patch actually introduces this issues. Bug 1517936 mentions similar exception already. But it makes it permafail instead of intermittent. This intermittent appeared in early January, could it be related to the first patches landed about compartments?

You might also want to try the patch in bug 1529306 as a fix for OOMs. Apparently some of the code was just kind of broken.

Thanks for the notice.
Here is a try run with the patch from bug 1529306:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=a3ede6d392cd911523ca7bce2b880b6ba0b246d6
(I just pushed it, so no results yet)

Attachment #9040399 - Attachment is obsolete: true

Alexandre, NI me if you want me to help with the Components.stack issue.

Yes, I think I would appreciate some help on this.

In comment 26, I linked to a try build that was artifact and so not picking up the platform fix.
I later pushed a non-artifact one, but the error was still around:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=8c4d31983d0b87cb21361f7e85112e3faca6602b

At the same time, I'm wondering if pgo builds are working correctly on try. For a long time it was impossible to build pgo correctly via the try syntax...

I rebased again the patch on today's m-c to ensure it really did not fixed the latest failure (browser_rules_user-agent-styles.js):
https://treeherder.mozilla.org/#/jobs?repo=try&revision=420d5283372a5aba3804b491af045ff4d38a5ba1

Here is another one, with a hacky workaround for callFunctionWithAsyncStack:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=00fed69c1c7b5d0d46cdfd4508c86b852e890b98

And a last one, with only freshCompartment removal to see if we can now remove the forced GC from WebConsole tests:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=f18a126f4487b7e5f1990e4ea2551dc6ef92ce11

Flags: needinfo?(jdemooij)

A few conclusions out of these try runs is that the forced GC is no longer useful. The web console tests now pass on Windows 7.
There is still an issue with browser_rules_user-agent-styles.js but it seems to be failing less frequently.
It is not clear if it fails more than on m-c, so here is a try run for m-c tip:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=0279c5021c1d0409909959d52db757283371c7c6

OK I'll try to look into this soon. A bit swamped right now but I should have time in a few days.

m-c try run
https://treeherder.mozilla.org/#/jobs?repo=try&revision=0279c5021c1d0409909959d52db757283371c7c6
43 runs, no failure.

try run with only freshCompartment removal:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=f18a126f4487b7e5f1990e4ea2551dc6ef92ce11
46 runs, 3 failures.

multi platforms try run:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=c6f3da848b7ab296aa8f568fcb4fb707a29caea1
There is suspicious intermittents on dt7 for linux 64 debug, but it also looks like existing ones.
bug 1511247 (the test is being disabled) and bug 1527224 (with a beginning of investigation)

I don't know what to conclude. It isn't clear if the 3 versus 0 means anything.
It may just slightly increase its frequency.
There is two bugs on file for browser_rules_user-agent-styles.js
bug 1517530 and bug 1530927 (new bug, looks like a duplicated)

I just pushed to try with a few forceShrinkingGC on all these intermittents, to see if this has any impact:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=bcbb52022a42d7f026faf6fa9c53fb3832fb2b80

We talked on IRC, Alexandre thinks we're good now.

Flags: needinfo?(jdemooij)

Yes. It looks like additional forceShrinkingGC on browser_rules_user-agent-styles.js prevent it from becoming slightly more intermittent. It may actually solve the existing intermittent!

The other linux debug intermittents looks like existing ones. Adding force GCs doesn't help.

So I'll move forward with workaround just for browser_rules_user-agent-styles.js

Pushed by apoirot@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/191a7f8a2816
Stop always loading DevTools modules in fresh compartments. r=jandem,nchevobbe

https://hg.mozilla.org/mozilla-central/rev/191a7f8a2816

Status: NEW → RESOLVED
Closed: 6 years ago
status-firefox67: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → Firefox 67

This change had a very significant impact on inspector opening (Up to 40% faster against complex documents).
While also speeding up some of inspector actions like expanding a DOM Element children or updating the inspector after a page reload (More than 40% faster).
The debugger is also faster to step in/out (20 to 30% faster).
Overall, the unittest asserting base RDP protocol performance is 20% faster.
The console is up to 15% faster to show object attributes when expanding an object.
Otherwise, almost all other tests report between 2 to 10% improvement on all DevTools.

== Change summary for alert #19714 (as of Thu, 28 Feb 2019 16:35:01 GMT) ==

Regressions:

16% damp custom.jsdebugger.stepOut.DAMP linux64-pgo-qr opt e10s stylo 416.96 -> 485.45
10% damp custom.jsdebugger.pause.DAMP osx-10-10 opt e10s stylo 662.50 -> 731.13
10% damp complicated.inspector.close.DAMP linux64-pgo-qr opt e10s stylo 44.59 -> 48.98

Improvements:

49% damp custom.inspector.collapseall.manychildren windows10-64 pgo e10s stylo 1.90 -> 0.96
49% damp custom.inspector.reload.DAMP windows10-64 pgo e10s stylo 839.22 -> 431.18
48% damp custom.inspector.reload.DAMP windows10-64-pgo-qr opt e10s stylo 826.09 -> 427.79
47% damp custom.inspector.collapseall.manychildren osx-10-10 opt e10s stylo 4.75 -> 2.50
46% damp custom.inspector.reload.DAMP osx-10-10 opt e10s stylo 1,997.74 -> 1,082.63
46% damp custom.inspector.collapseall.manychildren windows10-64-pgo-qr opt e10s stylo 1.86 -> 1.01
44% damp custom.inspector.reload.DAMP linux64 pgo e10s stylo 722.68 -> 408.22
43% damp custom.inspector.reload.DAMP linux64-pgo-qr opt e10s stylo 749.90 -> 426.19
39% damp custom.inspector.open.DAMP windows10-64 pgo e10s stylo 1,050.06 -> 635.55
39% damp custom.inspector.open.DAMP windows10-64-pgo-qr opt e10s stylo 1,046.71 -> 634.77
38% damp custom.inspector.collapseall.balanced windows10-64 pgo e10s stylo 14.09 -> 8.68
36% damp custom.inspector.collapseall.balanced linux64-pgo-qr opt e10s stylo 11.93 -> 7.62
34% damp custom.inspector.open.DAMP osx-10-10 opt e10s stylo 2,146.33 -> 1,410.82
34% damp custom.inspector.open.DAMP linux64-pgo-qr opt e10s stylo 956.24 -> 629.43
34% damp custom.inspector.collapseall.balanced linux64 pgo e10s stylo 11.50 -> 7.59
34% damp custom.inspector.open.DAMP linux64 pgo e10s stylo 917.91 -> 605.47
34% damp custom.inspector.collapseall.balanced windows10-64-pgo-qr opt e10s stylo 13.38 -> 8.85
30% damp custom.jsdebugger.stepIn.DAMP windows10-64 pgo e10s stylo 594.69 -> 415.91
30% damp custom.jsdebugger.stepIn.DAMP windows10-64-pgo-qr opt e10s stylo 580.45 -> 407.65
29% damp custom.inspector.manyrules.selectnode osx-10-10 opt e10s stylo 2,140.49 -> 1,510.46
28% damp custom.inspector.expandall.balanced linux64-pgo-qr opt e10s stylo 746.48 -> 535.35
27% damp custom.inspector.expandall.balanced windows10-64 pgo e10s stylo 760.99 -> 552.31
27% damp custom.inspector.expandall.manychildren osx-10-10 opt e10s stylo 471.67 -> 345.04
26% damp custom.inspector.expandall.balanced linux64 pgo e10s stylo 690.07 -> 507.24
26% damp custom.jsdebugger.stepIn.DAMP linux64 pgo e10s stylo 485.72 -> 357.40
26% damp custom.inspector.expandall.balanced windows10-64-pgo-qr opt e10s stylo 759.94 -> 561.43
24% damp custom.inspector.manyrules.selectnode windows10-64 pgo e10s stylo 928.38 -> 707.17
23% damp custom.jsdebugger.stepOut.DAMP osx-10-10 opt e10s stylo 1,128.49 -> 866.51
23% damp custom.inspector.expandall.manychildren windows10-64 pgo e10s stylo 214.11 -> 164.88
23% damp custom.jsdebugger.stepIn.DAMP linux64-pgo-qr opt e10s stylo 467.95 -> 360.86
23% damp custom.inspector.manyrules.selectnode linux64 pgo e10s stylo 794.51 -> 613.26
22% damp custom.inspector.manyrules.selectnode linux64-pgo-qr opt e10s stylo 832.78 -> 646.88
22% damp custom.inspector.manyrules.selectnode windows10-64-pgo-qr opt e10s stylo 914.27 -> 710.62
22% damp custom.inspector.expandall.balanced osx-10-10 opt e10s stylo 1,897.44 -> 1,481.08
21% damp custom.inspector.expandall.manychildren linux64 pgo e10s stylo 175.97 -> 138.20
21% damp server.protocoljs.DAMP linux64-pgo-qr opt e10s stylo 1,900.46 -> 1,507.64
21% damp custom.inspector.expandall.manychildren linux64-pgo-qr opt e10s stylo 190.13 -> 151.00
20% damp custom.inspector.expandall.manychildren windows10-64-pgo-qr opt e10s stylo 213.81 -> 170.08
19% damp server.protocoljs.DAMP osx-10-10 opt e10s stylo 2,641.68 -> 2,152.03
17% damp server.protocoljs.DAMP linux64 pgo e10s stylo 1,817.58 -> 1,507.78
16% damp custom.jsdebugger.stepIn.DAMP osx-10-10 opt e10s stylo 1,439.40 -> 1,205.94
15% damp console.objectexpand windows10-64 pgo e10s stylo 406.78 -> 345.97
15% damp custom.inspector.collapseall.manychildren linux64-pgo-qr opt e10s stylo 1.49 -> 1.27
15% damp console.objectexpand windows10-64-pgo-qr opt e10s stylo 407.27 -> 347.06
15% damp custom.inspector.collapseall.manychildren linux64 pgo e10s stylo 1.45 -> 1.24
14% damp panelsInBackground.reload.DAMP osx-10-10 opt e10s stylo 1,357.58 -> 1,161.33
12% damp console.objectexpand linux64-pgo-qr opt e10s stylo 397.20 -> 347.98
12% damp console.objectexpand linux64 pgo e10s stylo 387.76 -> 341.26
12% damp complicated.styleeditor.open.DAMP windows10-64 pgo e10s stylo 727.59 -> 642.65
11% damp complicated.styleeditor.open.DAMP windows10-64-pgo-qr opt e10s stylo 724.99 -> 644.43
10% damp complicated.styleeditor.open.DAMP linux64 pgo e10s stylo 739.65 -> 662.30
10% damp panelsInBackground.reload.DAMP windows10-64-pgo-qr opt e10s stylo 592.21 -> 531.35
9% damp custom.inspector.close.DAMP windows10-64 pgo e10s stylo 70.89 -> 64.32
9% damp custom.inspector.close.DAMP windows10-64-pgo-qr opt e10s stylo 71.63 -> 65.12
9% damp custom.inspector.manyrules.deselectnode windows10-64 pgo e10s stylo 194.75 -> 177.34
9% damp complicated.styleeditor.open.DAMP linux64-pgo-qr opt e10s stylo 769.71 -> 703.14
9% damp console.autocomplete windows10-64 pgo e10s stylo 978.31 -> 893.81
8% damp console.autocomplete windows10-64-pgo-qr opt e10s stylo 966.68 -> 887.66
8% damp console.autocomplete linux64-pgo-qr opt e10s stylo 943.92 -> 867.67
8% damp complicated.netmonitor.reload.DAMP osx-10-10 opt e10s stylo 5,805.05 -> 5,356.44
8% damp simple.styleeditor.open.DAMP windows10-64-pgo-qr opt e10s stylo 197.80 -> 182.82
8% damp complicated.netmonitor.requestsFinished.DAMP osx-10-10 opt e10s stylo 6,346.85 -> 5,870.01
8% damp custom.inspector.manyrules.deselectnode osx-10-10 opt e10s stylo 419.94 -> 388.43
7% damp custom.inspector.manyrules.deselectnode linux64-pgo-qr opt e10s stylo 182.01 -> 168.55
7% damp complicated.webconsole.reload.DAMP osx-10-10 opt e10s stylo 3,548.43 -> 3,289.61
7% damp complicated.webconsole.reload.DAMP linux64-pgo-qr opt e10s stylo 1,337.56 -> 1,243.28
7% damp complicated.netmonitor.reload.DAMP linux64-pgo-qr opt e10s stylo 1,904.83 -> 1,775.01
7% damp complicated.netmonitor.requestsFinished.DAMP windows10-64-pgo-qr opt e10s stylo 2,183.63 -> 2,036.18
7% damp custom.webconsole.reload.DAMP osx-10-10 opt e10s stylo 7,591.48 -> 7,080.86
7% damp complicated.netmonitor.requestsFinished.DAMP linux64 pgo e10s stylo 1,958.94 -> 1,829.13
7% damp osx-10-10 opt e10s stylo 435.63 -> 406.82
7% damp complicated.webconsole.reload.DAMP linux64 pgo e10s stylo 1,217.39 -> 1,137.15
7% damp console.autocomplete linux64 pgo e10s stylo 922.24 -> 861.56
7% damp simple.inspector.open.DAMP windows10-64 pgo e10s stylo 311.79 -> 291.42
6% damp complicated.netmonitor.reload.DAMP windows10-64-pgo-qr opt e10s stylo 2,000.15 -> 1,870.43
6% damp custom.inspector.manyrules.deselectnode windows10-64-pgo-qr opt e10s stylo 192.42 -> 180.00
6% damp complicated.netmonitor.reload.DAMP linux64 pgo e10s stylo 1,776.90 -> 1,663.66
6% damp complicated.styleeditor.reload.DAMP osx-10-10 opt e10s stylo 2,994.63 -> 2,808.80
6% damp windows10-64-pgo-qr opt e10s stylo 198.48 -> 186.49
6% damp complicated.inspector.open.DAMP windows10-64-pgo-qr opt e10s stylo 367.25 -> 345.28
6% damp complicated.inspector.open.DAMP linux64 pgo e10s stylo 360.11 -> 338.74
6% damp complicated.netmonitor.requestsFinished.DAMP windows10-64 pgo e10s stylo 2,188.61 -> 2,059.81
6% damp custom.inspector.manyrules.deselectnode linux64 pgo e10s stylo 169.70 -> 159.90
6% damp simple.inspector.open.DAMP windows10-64-pgo-qr opt e10s stylo 305.55 -> 288.09
6% damp complicated.inspector.open.DAMP windows10-64 pgo e10s stylo 369.94 -> 348.89
6% damp custom.jsdebugger.open.DAMP windows10-64-pgo-qr opt e10s stylo 924.72 -> 872.37
6% damp complicated.netmonitor.close.DAMP linux64 pgo e10s stylo 47.71 -> 45.03
6% damp custom.webconsole.open.DAMP osx-10-10 opt e10s stylo 1,017.22 -> 961.11
6% damp windows10-64 pgo e10s stylo 199.49 -> 188.50
5% damp custom.jsdebugger.open.DAMP windows10-64 pgo e10s stylo 916.90 -> 869.36
5% damp custom.jsdebugger.open.DAMP linux64 pgo e10s stylo 900.02 -> 854.94
5% damp custom.jsdebugger.stepOver.DAMP windows10-64 pgo e10s stylo 269.50 -> 256.07
5% damp simple.inspector.open.DAMP linux64 pgo e10s stylo 307.14 -> 291.88
5% damp console.bulklog windows10-64 pgo e10s stylo 113.86 -> 108.23
5% damp custom.jsdebugger.open.DAMP linux64-pgo-qr opt e10s stylo 937.12 -> 891.37
5% damp inspector.layout.open windows10-64-pgo-qr opt e10s stylo 327.99 -> 312.12
5% damp panelsInBackground.reload.DAMP linux64-pgo-qr opt e10s stylo 497.33 -> 473.98
5% damp complicated.netmonitor.exportHar linux64-pgo-qr opt e10s stylo 833.45 -> 794.82
5% damp simple.styleeditor.open.DAMP windows10-64 pgo e10s stylo 198.44 -> 189.24
4% damp custom.inspector.close.DAMP linux64 pgo e10s stylo 64.48 -> 61.58
4% damp console.bulklog windows10-64-pgo-qr opt e10s stylo 116.03 -> 110.86
4% damp inspector.layout.open linux64 pgo e10s stylo 336.93 -> 322.12
4% damp linux64 pgo e10s stylo 191.46 -> 183.03
4% damp custom.webconsole.open.DAMP linux64 pgo e10s stylo 494.15 -> 472.60
4% damp custom.jsdebugger.stepOver.DAMP linux64 pgo e10s stylo 264.25 -> 252.81
4% damp simple.inspector.open.DAMP linux64-pgo-qr opt e10s stylo 321.00 -> 307.12
4% damp custom.webconsole.open.DAMP windows10-64-pgo-qr opt e10s stylo 527.51 -> 505.17
4% damp linux64-pgo-qr opt e10s stylo 199.28 -> 190.85
4% damp custom.webconsole.open.DAMP windows10-64 pgo e10s stylo 533.30 -> 510.93
4% damp custom.jsdebugger.stepOver.DAMP linux64-pgo-qr opt e10s stylo 268.86 -> 257.61
4% damp simple.styleeditor.open.DAMP linux64-pgo-qr opt e10s stylo 213.72 -> 204.77
4% damp inspector.layout.open linux64-pgo-qr opt e10s stylo 350.27 -> 335.76
4% damp inspector.layout.open windows10-64 pgo e10s stylo 332.45 -> 318.70
4% damp simple.netmonitor.open.DAMP windows10-64-pgo-qr opt e10s stylo 211.19 -> 202.48
4% damp complicated.inspector.reload.DAMP osx-10-10 opt e10s stylo 3,333.00 -> 3,196.31
4% damp simple.styleeditor.open.DAMP linux64 pgo e10s stylo 209.48 -> 200.92
4% damp simple.netmonitor.open.DAMP windows10-64 pgo e10s stylo 213.85 -> 205.24
4% damp complicated.netmonitor.exportHar linux64 pgo e10s stylo 796.23 -> 764.16
4% damp inspector.mutations windows10-64 pgo e10s stylo 1,078.87 -> 1,037.60
4% damp complicated.inspector.reload.DAMP linux64 pgo e10s stylo 1,149.02 -> 1,105.13
4% damp inspector.mutations linux64-pgo-qr opt e10s stylo 998.32 -> 962.56
4% damp console.bulklog linux64 pgo e10s stylo 110.60 -> 106.64
3% damp complicated.webconsole.open.DAMP windows10-64-pgo-qr opt e10s stylo 275.46 -> 265.99
3% damp custom.webconsole.reload.DAMP linux64-pgo-qr opt e10s stylo 5,706.74 -> 5,517.55
3% damp inspector.mutations windows10-64-pgo-qr opt e10s stylo 1,077.77 -> 1,043.02
3% damp custom.webconsole.open.DAMP linux64-pgo-qr opt e10s stylo 508.06 -> 492.11
3% damp custom.webconsole.reload.DAMP windows10-64 pgo e10s stylo 5,808.73 -> 5,629.04
3% damp custom.webconsole.close.DAMP linux64-pgo-qr opt e10s stylo 79.92 -> 77.55
3% damp custom.webconsole.reload.DAMP windows10-64-pgo-qr opt e10s stylo 5,805.87 -> 5,635.17
3% damp cold.inspector.open.DAMP windows10-64-pgo-qr opt e10s stylo 429.78 -> 417.34
3% damp complicated.jsdebugger.reload.DAMP linux64 pgo e10s stylo 1,613.54 -> 1,569.57
3% damp custom.webconsole.close.DAMP windows10-64-pgo-qr opt e10s stylo 84.29 -> 82.15
2% damp complicated.jsdebugger.open.DAMP linux64 pgo e10s stylo 1,254.57 -> 1,223.29
2% damp custom.webconsole.reload.DAMP linux64 pgo e10s stylo 5,748.87 -> 5,615.42
2% damp simple.jsdebugger.open.DAMP windows10-64-pgo-qr opt e10s stylo 628.61 -> 614.60
2% damp simple.jsdebugger.open.DAMP linux64 pgo e10s stylo 627.83 -> 614.19
2% damp simple.jsdebugger.open.DAMP windows10-64 pgo e10s stylo 637.01 -> 623.38
2% damp simple.netmonitor.requestsFinished.DAMP windows10-64 pgo e10s stylo 143.26 -> 140.22

For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=19714

Depends on: 1532237
Depends on: 1532238
Blocks: 1540694
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: