Perma browser_aboutdebugging_devtoolstoolbox_reload.js | leaked 2 window(s) until shutdown when Gecko 70 merges to Beta on 2019-08-26
Categories
(DevTools :: about:debugging, defect)
Tracking
(firefox-esr60 unaffected, firefox-esr68 unaffected, firefox68 unaffected, firefox69 unaffected, firefox70+ verified)
Tracking | Status | |
---|---|---|
firefox-esr60 | --- | unaffected |
firefox-esr68 | --- | unaffected |
firefox68 | --- | unaffected |
firefox69 | --- | unaffected |
firefox70 | + | verified |
People
(Reporter: CosminS, Assigned: jdescottes)
References
Details
(Keywords: regression)
Attachments
(2 files)
[Tracking Requested - why for this release]:
Central as beta simulation: https://treeherder.mozilla.org/#/jobs?repo=try&selectedJob=256344087&revision=7c891a7092619e9f2bf3abb68b4542062420e0d4&searchStr=devtools&group_state=expanded
Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=256344087&repo=try&lineNumber=7257
Even though the initial cause for these failures in tree (Bug 1354679) was backed out here https://hg.mozilla.org/mozilla-central/rev/cbac7fc3b4472c9f0f1a0d5c9be5c0f55f7419dd and was merged to central a few hrs later, these failures still happen on beta sims.
The tests are green on integration trees.
Our hunch is Bug 1560178 and the changes here: https://hg.mozilla.org/mozilla-central/rev/6071c24be566a2b3f2f98152456a540e9e893377
Gijs, Aryx could you please take a look over this and see why this happens? Thanks.
Comment 1•5 years ago
|
||
It looks like in https://bugzilla.mozilla.org/show_bug.cgi?id=1354679#c32 you backed out only the "tweak pause overlay" changeset, but bug 1544828 was filed immediately after 6363111ee8ce ("Automatically display the PausedDebuggerOverlay when the debugger is paused.") landed, and that doesn't look like it's been backed out?
Comment 2•5 years ago
|
||
And e.g. https://treeherder.mozilla.org/#/jobs?repo=autoland&fromchange=cbac7fc3b4472c9f0f1a0d5c9be5c0f55f7419dd&searchStr=devtools&selectedJob=256278810 and https://treeherder.mozilla.org/#/jobs?repo=autoland&fromchange=cbac7fc3b4472c9f0f1a0d5c9be5c0f55f7419dd&searchStr=devtools&selectedJob=256273966 are after the backout and still orange. I don't know if the frequency has really gone down, it'll probably be hard-ish to tell until the weekend is over and normal push frequency resumes.
Comment hidden (Intermittent Failures Robot) |
Comment 4•5 years ago
|
||
Fixed by backout of https://hg.mozilla.org/mozilla-central/rev/6363111ee8ce from bug 1354679.
Comment 5•5 years ago
|
||
Hi, do you have an intermittent view of the failures?
https://treeherder.mozilla.org/intermittent-failures.html I'd like to see which platforms (or all) it was failing on and under what modes presumably debug...
Reporter | ||
Comment 6•5 years ago
|
||
Jason, this was filled as a beta-sim bug and is normally stared on tree as expected fail so no intermittent failures view.
But here's a failure rate from Bug 1544828 which is I think the same only for integration trees: https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?startday=2019-07-08&endday=2019-07-15&tree=trunk&bug=1544828
It fails mostly on linux64 debug: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=255650432&repo=autoland&lineNumber=7423 and sometimes on macosx1014-64 debug: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=256217833&repo=autoland&lineNumber=8097
Marking fixed from comment 4.
Comment 8•5 years ago
|
||
This is still occurring on recent beta simulations:
https://treeherder.mozilla.org/#/jobs?repo=try&resultStatus=pending%2Crunning%2Ctestfailed%2Cbusted%2Cexception&revision=9bdb6707b69111f77ea9ad48f83f6f57d147b394&group_state=expanded&selectedJob=257522326
Aryx could the recent occurrences be from Bug 1497197?
Updated•5 years ago
|
Reporter | ||
Comment 9•5 years ago
|
||
This looks to be from https://hg.mozilla.org/integration/autoland/rev/63e23c9ed252 with the changes from Bug 1354679 being repushed on tree. Also the culprit must be in this pushlog https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=dc32c9061a0a335dd582845703cd742c55be9c85&tochange=872c3c79f9060ac27b6b79435e4be869bb040e91 cause in yesterdays beta-sims the tests were green: https://tinyurl.com/yxhch8gj
Comment 10•5 years ago
|
||
Bugbug thinks this bug is a regression, but please revert this change in case of error.
Updated•5 years ago
|
Comment 11•5 years ago
|
||
Comment 12•5 years ago
|
||
Hi, I'm a bit confused... I don't see any failures for this test on TRY or Central. I also can't reproduce this error locally in debug mode on osx. The test is extremely slow (requestLongerTimeout(5);
) so it is very tough to get meaningful data...
Do you have a good report where I can see the failures?
Comment 13•5 years ago
|
||
I had a try run last week with 50+ runs and no leaks, which seemed to confirm that the overlay was not to blame. comment
Here is a new try run on top of master with the overlay. I'm only testing the aboutdebugging directory because I want to have as many runs as possible:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=5f16ad4345724a90e636a40e1e61032dadb8013b
https://treeherder.mozilla.org/#/jobs?repo=try&revision=38af0913f6097f5991568d814b1e58d5395fd90e
Comment 14•5 years ago
|
||
(In reply to Jason Laster [:jlast] from comment #12)
Hi, I'm a bit confused... I don't see any failures for this test on TRY or Central
As comment #0 points out, this is about what happens when central merges to beta. It doesn't look like your trypushes contain the relevant changes to test that configuration. You can probably import something like https://hg.mozilla.org/try/rev/7c891a7092619e9f2bf3abb68b4542062420e0d4 to do the same kind of tests.
It's likely something about the debugger or what is being debugged is nightly-only and so this test behaves differently in a beta configuration.
Comment 15•5 years ago
|
||
You can get the beta simulation by running the command from https://wiki.mozilla.org/Sheriffing/How_To/Beta_simulations#TRUNK_AS_EARLY_BETA but append --no-push
to not run everything but hg commit
the changes and run ./mach try chooser
to pick the needed ones.
Comment 16•5 years ago
|
||
The test that is causing the issues was recently disabled for linux debug link. Do you think it is reasonable to disable for debug? The test is already using requestLongerTimeout(5);
and is known for timeouts and OOMs, i'm not too surprised that it would leak in a failure.
Updated•5 years ago
|
Assignee | ||
Comment 17•5 years ago
|
||
(In reply to Jason Laster [:jlast] from comment #16)
The test that is causing the issues was recently disabled for linux debug link.
Yes it was recently disabled on Linux debug. The perma fail for this test on debug mode started with Bug 1354679 (pause overlay). We should investigate why the debugger now leaks windows on reload.
Do you think it is reasonable to disable for debug?
We tried to do that in https://bugzilla.mozilla.org/show_bug.cgi?id=1544828, but it made the next test in the suite timeout (no idea why).
Either we skip this test and the next one in debug (potentially others? to confirm with a try push), or we fix the debugger leak that started with Bug 1354679.
Assignee | ||
Comment 18•5 years ago
|
||
I also don't know if we really have an issue specific to the beta simulation or not here. There has been movement on both Bug 1354679 and Bug 1544828, so it's hard to say at the moment. I kicked a few try runs, we'll see how it goes:
Updated•5 years ago
|
Assignee | ||
Comment 19•5 years ago
|
||
I will take the bug for now.
Assignee | ||
Comment 20•5 years ago
|
||
Quick update to say that there is definitely something wrong linked with this test + beta simulation:
- permafail with beta simulation: https://treeherder.mozilla.org/#/jobs?repo=try&revision=a6975d7eab5809347c25775940f971500d80947c
- no failure on central for the same changeset: https://treeherder.mozilla.org/#/jobs?repo=try&revision=713bbd2fd9d06b7ee26176c19b1a498a1abb68e4
Assignee | ||
Comment 21•5 years ago
|
||
This avoids intermittent leaks in aboutdebugging tests relying on about:devtools-toolbox
Assignee | ||
Comment 22•5 years ago
|
||
Depends on D39174
Seems to fix the perma fail on beta
Assignee | ||
Comment 23•5 years ago
|
||
The patches attached here should fix the issue on beta osx debug:
https://treeherder.mozilla.org/#/jobs?repo=try&selectedJob=258088202&revision=d6d2e12be3c372c849181aa45c952ded5bd6db2d
The approach here is to wait for toolbox requests to be finished before reloading the toolbox and shutting down about:devtools-toolbox.
I still don't really know why the debugger patch regressed this (and I checked that the beta simulations were only perma-failing with Bug 1354679). My guess is that the debugger's initialization is not properly waiting for everything to be done before emitting the "ready" event and that should ideally be fixed. But for now that should be enough.
Assignee | ||
Comment 24•5 years ago
|
||
Try push on central without beta sim is green as well https://treeherder.mozilla.org/#/jobs?repo=try&revision=2ebfc9b4623988cfd3550e679aa1ec4c7b355725
Assignee | ||
Updated•5 years ago
|
Comment 25•5 years ago
|
||
Pushed by jdescottes@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/c314c3bab4f2 Wait for debugger-client requests to settle before closing the toolbox r=remote-debugging-reviewers,Ola,daisuke https://hg.mozilla.org/integration/autoland/rev/6cd0d9de893d Wait for requests to settle before reloading about:devtools-toolbox in tests r=remote-debugging-reviewers,Ola,daisuke
Assignee | ||
Updated•5 years ago
|
Comment 26•5 years ago
|
||
bugherder |
https://hg.mozilla.org/mozilla-central/rev/c314c3bab4f2
https://hg.mozilla.org/mozilla-central/rev/6cd0d9de893d
Reporter | ||
Comment 27•5 years ago
|
||
Fix verified in today's beta-sim: https://treeherder.mozilla.org/#/jobs?repo=try&resultStatus=success%2Ctestfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel%2Crunnable&revision=2d55d46e818c791096b9c8c2e7114568e072e667&searchStr=devtools&group_state=expanded&selectedJob=258687988
Updated•5 years ago
|
Description
•