Closed Bug 1515057 Opened 6 years ago Closed 5 years ago

Intermittent leakcheck | default 1908/2012 bytes leaked (Mutex, PollableEvent, ReentrantMonitor, nsAStreamCopier, nsPipe, ...)

Categories

(Core :: DOM: Security, defect, P2)

Product:
Core
Component:
DOM: Security
Type:
defect

Tracking

()

Status:
RESOLVED WORKSFORME

People

(Reporter: intermittent-bug-filer, Assigned: tnguyen, NeedInfo)

References

(Regression)

Details

(Keywords: intermittent-failure, regression, Whiteboard: [domsecurity-intermittent][stockwell fixed:backout])

Filed by: cbrindusan [at] mozilla.com https://treeherder.mozilla.org/logviewer.html#?job_id=217646377&repo=mozilla-central https://queue.taskcluster.net/v1/task/R0M6LtJ5QIyp9Ysc4Lu-5A/runs/0/artifacts/public/logs/live_backing.log 11:25:25 INFO - Browser exited with return code 0 11:25:25 INFO - PROCESS LEAKS c:\users\task_1545131100\appdata\local\temp\tmptnuxzq.mozrunner\runtests_leaks_7520.log 11:25:25 INFO - leakcheck | Processing log file c:\users\task_1545131100\appdata\local\temp\tmptnuxzq.mozrunner\runtests_leaks_7520.log for scope /referrer-policy/unsafe-url/http-rp 11:25:25 INFO - TEST-INFO | leakcheck | default process: leak threshold set at 0 bytes 11:25:25 INFO - TEST-INFO | leakcheck | plugin process: leak threshold set at 0 bytes 11:25:25 INFO - TEST-INFO | leakcheck | tab process: leak threshold set at 10000 bytes 11:25:25 INFO - TEST-INFO | leakcheck | geckomediaplugin process: leak threshold set at 20000 bytes 11:25:25 INFO - TEST-INFO | leakcheck | gpu process: leak threshold set at 0 bytes 11:25:25 INFO - TEST-INFO | leakcheck | rdd process: leak threshold set at 400 bytes 11:25:25 INFO - leakcheck | Processing leak log file c:\users\task_1545131100\appdata\local\temp\tmptnuxzq.mozrunner\runtests_leaks_7520.log 11:25:25 INFO - 11:25:25 INFO - == BloatView: ALL (cumulative) LEAK AND BLOAT STATISTICS, default process 9068 11:25:25 INFO - 11:25:25 INFO - |<----------------Class--------------->|<-----Bytes------>|<----Objects---->| 11:25:25 INFO - | | Per-Inst Leaked| Total Rem| 11:25:25 INFO - 0 |TOTAL | 43 1908| 3601834 12| 11:25:25 INFO - 515 |Mutex | 80 240| 2841 3| 11:25:25 INFO - 758 |PollableEvent | 48 48| 1 1| 11:25:25 INFO - 814 |ReentrantMonitor | 40 40| 672 1| 11:25:25 INFO - 1316 |nsAStreamCopier | 40 40| 332 1| 11:25:25 INFO - 1648 |nsPipe | 208 208| 518 1| 11:25:25 INFO - 1649 |nsPipeInputStream | 136 136| 518 1| 11:25:25 INFO - 1703 |nsSocketTransport | 824 824| 13 1| 11:25:25 INFO - 1704 |nsSocketTransportService | 352 352| 1 1| 11:25:25 INFO - 1715 |nsStringBuffer | 12 12| 79835 1| 11:25:25 INFO - 1760 |nsTArray_base | 8 8| 1606026 1| 11:25:25 INFO - 11:25:25 INFO - nsTraceRefcnt::DumpStatistics: 1894 entries 11:25:25 INFO - TEST-INFO | leakcheck | default leaked 3 Mutex 11:25:25 INFO - TEST-INFO | leakcheck | default leaked 1 PollableEvent 11:25:25 INFO - TEST-INFO | leakcheck | default leaked 1 ReentrantMonitor 11:25:25 INFO - TEST-INFO | leakcheck | default leaked 1 nsAStreamCopier 11:25:25 INFO - TEST-INFO | leakcheck | default leaked 1 nsPipe 11:25:25 INFO - TEST-INFO | leakcheck | default leaked 1 nsPipeInputStream 11:25:25 INFO - TEST-INFO | leakcheck | default leaked 1 nsSocketTransport 11:25:25 INFO - TEST-INFO | leakcheck | default leaked 1 nsSocketTransportService 11:25:25 INFO - TEST-INFO | leakcheck | default leaked 1 nsStringBuffer 11:25:25 INFO - TEST-INFO | leakcheck | default leaked 1 nsTArray_base 11:25:25 INFO - TEST-UNEXPECTED-FAIL | leakcheck | default 1908 bytes leaked (Mutex, PollableEvent, ReentrantMonitor, nsAStreamCopier, nsPipe, ...)
Whiteboard: [domsecurity-intermittent]

Update: Just saw that bug 1525640 was backed out because of this: https://bugzilla.mozilla.org/show_bug.cgi?id=1525640#c9, so canceling the needinfo and all these failures should be fixed by commit d9ee5458ce46d43e760fe81dffe866673080b78e

After Bug 1525640 landed here this started to spike in failure rate: https://treeherder.mozilla.org/#/jobs?repo=autoland&searchStr=os%2Cx%2C10.10%2Cdebug%2Cweb%2Cplatform%2Ctests%2Cwith%2Ce10s%2Ctest-macosx64%2Fdebug-web-platform-tests-e10s-16%2Cw-e10s%28wpt16%29&tochange=efdd32c00dc61f9d8b377a78b9955b2d842f28fd&fromchange=d562b158decc56ba2b9b270f4857f75458534d53&group_state=expanded&selectedJob=240125123

It only fails on OSX debug but from 12th of April it has 174 failures: https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?startday=2019-04-07&endday=2019-04-14&tree=trunk&bug=1515057
Looking through the logs it fails on different tests so it's not related to one specific test:
/referrer-policy/strict-origin/meta-referrer/same-origin/http-https/xhr-request/no-redirect/upgrade-protocol.http.html
/css/css-transitions/parsing/transition-timing-function-valid.html
/css/css-exclusions/wrap-through-001.html
/referrer-policy/origin/http-rp/same-origin/http-https/xhr-request/swap-origin-redirect/generic.http.html
/ambient-light/idlharness.https.window.html

Valentin please take a look over here as soon as you can. Thank you. Recent failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=240058723&repo=mozilla-central&lineNumber=41500

Flags: needinfo?(valentin.gosu)
Flags: needinfo?(valentin.gosu)
Whiteboard: [domsecurity-intermittent][stockwell disable-recommended] → [domsecurity-intermittent]
Whiteboard: [domsecurity-intermittent][stockwell disable-recommended] → [domsecurity-intermittent]
Whiteboard: [domsecurity-intermittent][stockwell disable-recommended] → [domsecurity-intermittent]
Whiteboard: [domsecurity-intermittent][stockwell disable-recommended] → [domsecurity-intermittent]

This still has a very high frequency rate. There are 579 total failures in the last 7 days on osx-10-10 debug

Recent failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=241365571&repo=autoland&lineNumber=31684

04:05:38 INFO - TEST-START | /referrer-policy/strict-origin/http-rp/same-origin/http-https/img-tag/swap-origin-redirect/upgrade-protocol.http.html
04:05:38 INFO - Closing window 6442450945
04:05:38 INFO - PID 1669 | [Parent 1669, Main Thread] WARNING: '!mSelection', file /builds/worker/workspace/build/src/editor/libeditor/EditorBase.cpp, line 4889
04:05:38 INFO - PID 1669 | [Parent 1669, Main Thread] WARNING: '!editActionData.CanHandle()', file /builds/worker/workspace/build/src/editor/libeditor/EditorBase.cpp, line 1292

04:05:43 INFO - TEST-INFO | leakcheck | default leaked 1 nsSocketTransportService
04:05:43 INFO - TEST-INFO | leakcheck | default leaked 1 nsStringBuffer
04:05:43 INFO - TEST-INFO | leakcheck | default leaked 1 nsTArray_base
04:05:43 INFO - TEST-UNEXPECTED-FAIL | leakcheck | default 2012 bytes leaked (Mutex, PollableEvent, ReentrantMonitor, nsAStreamCopier, nsPipe, ...)
04:05:43 INFO -
04:05:43 INFO - leakcheck | Processing leak log file /var/folders/jy/18dmp9t92px8pdvdg_81xk8000000w/T/tmpxl8mus.mozrunner/runtests_leaks_699_tab_pid1671.log
04:05:43 INFO -
04:05:43 INFO - == BloatView: ALL (cumulative) LEAK AND BLOAT STATISTICS, tab process 1671

This appears to be from 1501108, Eugen please take a look. The leaks are in different tests, we cannot disable it.

Recent occurences are on the 15th from here: https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=1936dde5f34c98c7eaca6d53d9826e3b40f1134c&selectedJob=240466557 the bug got backed out and then relanded

causing the increase here: https://treeherder.mozilla.org/#/jobs?repo=autoland&searchStr=web%2Cplatform&tochange=8f2d511ad49f2a361a1fe7d4af1b204c13a05474&fromchange=144a2491cbd523c2d4d3bff136165c8a97d00f40&group_state=expanded&selectedJob=240997767

Flags: needinfo?(esawin)
Regressed by: 1501108
Whiteboard: [domsecurity-intermittent] → [domsecurity-intermittent][stockwell needswork:owner]
Keywords: regression

There are 815 total failures in the last 7 days on osx-10-10 debug.

Recent failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=241683422&repo=mozilla-inbound&lineNumber=10340

11:16:32 INFO - TEST-START | /css/selectors/focus-within-display-none-001.html
11:16:36 INFO - TEST-UNEXPECTED-FAIL | leakcheck | default 2012 bytes leaked (Mutex, PollableEvent, ReentrantMonitor, nsAStreamCopier, nsPipe, ...)
11:16:36 INFO -
11:16:36 INFO - leakcheck | Processing leak log file /var/folders/7g/sw1yvcpd3ss3brv9_00zh8h800000w/T/tmpHVDu4Z.mozrunner/runtests_leaks_1914_tab_pid2236.log
11:16:36 INFO -
11:16:36 INFO - == BloatView: ALL (cumulative) LEAK AND BLOAT STATISTICS, tab process 2236
11:16:36 INFO -
11:16:36 INFO - |<----------------Class--------------->|<-----Bytes------>|<----Objects---->|
11:16:36 INFO - | | Per-Inst Leaked| Total Rem|
11:16:36 INFO - 0 |TOTAL | 42 0| 79121 0|
11:16:36 INFO -
11:16:37 INFO - nsTraceRefcnt::DumpStatistics: 775 entries
11:16:37 INFO - TEST-PASS | leakcheck | tab no leaks detected!
11:16:37 INFO - leakcheck | Processing leak log file /var/folders/7g/sw1yvcpd3ss3brv9_00zh8h800000w/T/tmpHVDu4Z.mozrunner/runtests_leaks_1914_tab_pid2237.log
11:16:37 INFO -
11:16:37 INFO - == BloatView: ALL (cumulative) LEAK AND BLOAT STATISTICS, tab process 2237
11:16:37 INFO -
11:16:37 INFO - |<----------------Class--------------->|<-----Bytes------>|<----Objects---->|
11:16:37 INFO - | | Per-Inst Leaked| Total Rem|
11:16:37 INFO - 0 |TOTAL | 36 0| 71619 0|
11:16:37 INFO -
11:16:37 INFO - nsTraceRefcnt::DumpStatistics: 861 entries
11:16:37 INFO - TEST-PASS | leakcheck | tab no leaks detected!
11:16:37 INFO - leakcheck | Processing leak log file /var/folders/7g/sw1yvcpd3ss3brv9_00zh8h800000w/T/tmpHVDu4Z.mozrunner/runtests_leaks_1914_tab_pid2238.log

Whiteboard: [domsecurity-intermittent][stockwell disable-recommended] → [domsecurity-intermittent][stockwell needwork:owner]

There are 915 failures in the last 7 days. All of them on osx-10-10 debug
:ckerschb , can you please take a look?

Flags: needinfo?(ckerschb)
Whiteboard: [domsecurity-intermittent][stockwell disable-recommended] → [domsecurity-intermittent]

James, can you drive this to get the situation resolved, please?

Flags: needinfo?(snorp)

The only thing I see that could be "leaking" from bug 1501108 is the added origin attribute, but I don't see how.

Flags: needinfo?(esawin)
Summary: Intermittent leakcheck | default 1908 bytes leaked (Mutex, PollableEvent, ReentrantMonitor, nsAStreamCopier, nsPipe, ...) → Intermittent leakcheck | default 1908/2012 bytes leaked (Mutex, PollableEvent, ReentrantMonitor, nsAStreamCopier, nsPipe, ...)

:baku, do you have ideas here? I can't see any obvious reason why bug 1501108 would trigger a leak...

Flags: needinfo?(snorp) → needinfo?(amarchesini)

(In reply to Narcis Beleuzu [:NarcisB] from comment #28)

There are 915 failures in the last 7 days. All of them on osx-10-10 debug
:ckerschb , can you please take a look?

Maybe Baku has some thoughts because of bug 1501108. Generally, the test in question is selectors/focus-within-display-none-001.html, which seems to be a CSS test, right? Maybe we can also consult some CSS folks to help us out?

Flags: needinfo?(ckerschb)
Whiteboard: [domsecurity-intermittent][stockwell disable-recommended] → [domsecurity-intermittent]
Whiteboard: [domsecurity-intermittent][stockwell disable-recommended] → [domsecurity-intermittent]

This still fails massively, 1911 total failures in the last 7 days. https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?startday=2019-04-22&endday=2019-04-29&tree=trunk&bug=1515057
Christoph, James could you needinfo someone who can help with this?

Flags: needinfo?(snorp)
Flags: needinfo?(ckerschb)
Status: NEW → ASSIGNED

Looks like Thomas is looking at this...

Flags: needinfo?(snorp)
Whiteboard: [domsecurity-intermittent][stockwell disable-recommended] → [domsecurity-intermittent]

(In reply to Christoph Kerschbaumer [:ckerschb] from comment #42)

Thomas, it seems the following referrer-policy test:

http://web-platform.test:8000/referrer-policy/strict-origin-when-cross-origin/http-rp/same-origin/http-https/img-tag/keep-origin-redirect/upgrade-protocol.http.html

is causing issues, see also:
https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&resultStatus=testfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel%2Crunnable&group_state=expanded&revision=f6766ba4ac77c6757c4e4db7598a2e685f23fcf6&searchStr=10.10%2Cdebug%2Cwpt&selectedJob=243449748

Can you take a look please?

I don't have any glue about the root cause at the moment, because the leak occurs randomly to all modules in wpt test (not only my test).
But I could see the rate of this leak downed very low last 2 days. Did we back out anything? Maybe we could have better glue from that.
In fact, this kind of leak existed for 1 year ago after (see : bug 1436244) after we enable wpt leak check, but low rate.

Flags: needinfo?(tnguyen) → needinfo?(snorp)

It has gone down because bug 1501108 got backed out which had triggered the spike when it landed.

Whiteboard: [domsecurity-intermittent][stockwell disable-recommended] → [domsecurity-intermittent][stockwell fixed:backout]

No leak after 23 days, I would like to close it

Status: ASSIGNED → RESOLVED
Closed: 5 years ago
Resolution: --- → WORKSFORME
Has Regression Range: --- → yes
You need to log in before you can comment on or make changes to this bug.