1635776 - Intermittent OSError: IO Completion Port failed to signal process shutdown

Reporter

Description

•

4 years ago

treeherder

Filed by: rgurzau [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=301014112&repo=mozilla-central
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/X-DK4v-VS3CdlLwl5Zq1eA/runs/0/artifacts/public/logs/live_backing.log

[task 2020-05-06T13:02:15.380Z] 13:02:15 [task 2020-05-06T13:02:15.380Z] 13:02:15 [task 2020-05-06T13:02:15.380Z] 13:02:15 [task 2020-05-06T13:02:15.380Z] 13:02:15 [task 2020-05-06T13:02:15.380Z] 13:02:15 [task 2020-05-06T13:02:15.381Z] 13:02:15 [task 2020-05-06T13:02:15.381Z] 13:02:15 [task 2020-05-06T13:02:15.381Z] 13:02:15 [task 2020-05-06T13:02:15.381Z] 13:02:15 [task 2020-05-06T13:02:15.381Z] 13:02:15 [task 2020-05-06T13:02:15.381Z] 13:02:15 [task 2020-05-06T13:02:15.381Z] 13:02:15 [task 2020-05-06T13:02:15.382Z] 13:02:15 [task 2020-05-06T13:02:15.382Z] 13:02:15 [task 2020-05-06T13:02:15.382Z] 13:02:15 [task 2020-05-06T13:02:15.382Z] 13:02:15 [task 2020-05-06T13:02:15.382Z] 13:02:15 [task 2020-05-06T13:02:15.382Z] 13:02:15 [task 2020-05-06T13:02:15.382Z] 13:02:15 [task 2020-05-06T13:02:15.382Z] 13:02:15 [task 2020-05-06T13:02:15.383Z] 13:02:15 [task 2020-05-06T13:02:15.383Z] 13:02:15 [task 2020-05-06T13:02:15.464Z] 13:02:15 [task 2020-05-06T13:02:15.464Z] 13:02:15 [task 2020-05-06T13:02:15.464Z] 13:02:15 [task 2020-05-06T13:02:15.465Z] 13:02:15 [task 2020-05-06T13:02:15.473Z] 13:02:15 [task 2020-05-06T13:02:15.473Z] [task 2020-05-06T13:02:15.474Z] 13:02:15 ERROR - Traceback (most recent call last):
INFO - File "C:\Users\task_1588767109\build\tests\raptor\raptor\raptor.py", line 201, in <module>
INFO - main()
INFO - File "C:\Users\task_1588767109\build\tests\raptor\raptor\raptor.py", line 148, in main
INFO - success = raptor.run_tests(raptor_test_list, raptor_test_names)
INFO - File "C:\Users\task_1588767109\build\tests\raptor\raptor\perftest.py", line 373, in run_tests
INFO - self.run_test(test, timeout=int(test.get("page_timeout")))
INFO - File "C:\Users\task_1588767109\build\tests\raptor\raptor\webextension\desktop.py", line 99, in run_test
INFO - self.__run_test_cold(test, timeout)
INFO - File "C:\Users\task_1588767109\build\tests\raptor\raptor\webextension\desktop.py", line 175, in __run_test_cold
INFO - self.wait_for_test_finish(test, timeout, self.process_exists)
INFO - File "C:\Users\task_1588767109\build\tests\raptor\raptor\webextension\base.py", line 128, in wait_for_test_finish
INFO - self.control_server.wait_for_quit()
INFO - File "C:\Users\task_1588767109\build\tests\raptor\raptor\control_server.py", line 425, in wait_for_quit
INFO - self.browser_proc.wait(timeout)
INFO - File "C:\Users\task_1588767109\build\venv\lib\site-packages\mozprocess\processhandler.py", line 948, in wait
INFO - self.returncode = self.proc.wait()
INFO - File "C:\Users\task_1588767109\build\venv\lib\site-packages\mozprocess\processhandler.py", line 260, in wait
INFO - self.returncode = self._custom_wait(timeout=timeout)
INFO - File "C:\Users\task_1588767109\build\venv\lib\site-packages\mozprocess\processhandler.py", line 587, in _custom_wait
INFO - raise OSError("IO Completion Port failed to signal process shutdown")
INFO - OSError: IO Completion Port failed to signal process shutdown
ERROR - Return code: 1
WARNING - setting return code to 1
INFO - Copying Raptor results to upload dir:
INFO - C:\Users\task_1588767109\build\blobber_upload_dir\perfherder-data.json
INFO - Copying raptor results from C:\Users\task_1588767109\build\raptor.json to C:\Users\task_1588767109\build\blobber_upload_dir\perfherder-data.json
13:02:15 CRITICAL - Error copying results C:\Users\task_1588767109\build\raptor.json to upload dir C:\Users\task_1588767109\build\blobber_upload_dir\perfherder-data.json
INFO - [Errno 2] No such file or directory: u'C:\Users\task_1588767109\build\raptor.json'

Comment hidden (Intermittent Failures Robot)

Greg Mierzwinski [:sparky]

Comment 5

•

4 years ago

Looks like this is a problem that's exclusive to windows (reference hardware).

Comment hidden (Intermittent Failures Robot)

Greg Mierzwinski [:sparky]

Comment 7

•

4 years ago

Moving to mozbase since this looks like a mozprocess failure on windows 10-64 ref 2017

Component: Raptor → Mozbase

Comment hidden (Intermittent Failures Robot)

BugBot [:suhaib / :marco/ :calixte]

Comment 19

•

4 years ago

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit auto_nag documentation.

Status: NEW → RESOLVED

Closed: 4 years ago

Resolution: --- → INCOMPLETE

Alexandru Michis [:malexandru]

Comment 20

•

4 years ago

Recent failure log: https://treeherder.mozilla.org/logviewer?job_id=330586107&repo=mozilla-central&lineNumber=1276

Status: RESOLVED → REOPENED

Resolution: INCOMPLETE → ---

Comment hidden (Intermittent Failures Robot)

Andreea Pavel [:apavel]

Comment 22

•

4 years ago

Recent failures here are browsertime, should we open a separate bug or is this one fine?

[task 2021-02-24T03:10:23.280Z] 03:10:23 INFO - raptor-mitmproxy Info: Verified mitmproxy CA certificate is installed in Firefox
[task 2021-02-24T03:10:23.280Z] 03:10:23 INFO - raptor-perftest Info: Playback recording information not available
[task 2021-02-24T03:10:23.281Z] 03:10:23 INFO - raptor-browsertime Info: test: {'expected_browser_cycles': 25, u'alert_threshold': u'2.0', u'playback_pageset_manifest': u'mitm5-linux-firefox-paypal.manifest', u'browser_cycles': u'25', 'here': u'C:\Users\task_1613996340\build\tests\raptor\raptor\tests\tp6\desktop', u'measure': [u'fnbpaint', u'fcp', u'dcf', u'loadtime'], 'path': u'C:\Users\task_1613996340\build\tests\raptor\raptor\tests\tp6\desktop\paypal', 'manifest_relpath': u'tests\tp6\desktop\browsertime-tp6.ini', 'cold': True, u'unit': u'ms', u'use_live_sites': u'false', u'alert_on': [u'fcp', u'loadtime'], u'type': u'pageload', 'name': u'paypal', u'playback_recordings': u'mitm5-linux-firefox-paypal.mp', u'page_cycles': 2, u'apps': u'firefox, chrome, chromium', u'test_url': u'https://www.paypal.com/myaccount/summary/', 'manifest': u'C:\Users\task_1613996340\build\tests\raptor\raptor\tests\tp6\desktop\browsertime-tp6.ini', 'ancestor_manifest': 'raptor.ini', u'page_timeout': u'60000', u'playback_version': u'5.1.1', u'playback': u'mitmproxy', 'browser_cycle': 1, 'expected': 'pass', u'lower_is_better': True, 'relpath': u'tests\tp6\desktop\paypal'}
[task 2021-02-24T03:10:23.281Z] 03:10:23 INFO - raptor-browsertime-desktop Info: binary_path: C:\Users\task_1613996340\build\application\firefox\firefox.exe
[task 2021-02-24T03:10:23.281Z] 03:10:23 INFO - raptor-browsertime Info: Using Firefox Window Recorder for videos
[task 2021-02-24T03:10:23.281Z] 03:10:23 INFO - raptor-browsertime Info: timeout (s): 60000
[task 2021-02-24T03:10:23.281Z] 03:10:23 INFO - raptor-browsertime Info: browsertime cwd: C:\Users\task_1613996340\build
[task 2021-02-24T03:10:23.281Z] 03:10:23 INFO - raptor-browsertime Info: browsertime cmd: C:/Users/task_1613996340/fetches/node/node.exe C:/Users/task_1613996340/fetches/browsertime/node_modules/browsertime/bin/browsertime.js --firefox.geckodriverPath C:/Users/task_1613996340/fetches/geckodriver.exe C:\Users\task_1613996340\build\tests\raptor\raptor\browsertime....\browsertime\browsertime_pageload.js --viewPort 1024x768 --browser firefox --firefox.binaryPath C:\Users\task_1613996340\build\application\firefox\firefox.exe --browsertime.page_cycles 2 --browsertime.url https://www.paypal.com/myaccount/summary/ --browsertime.page_cycle_delay 1000 --browsertime.post_startup_delay 30000 --firefox.profileTemplate c:\users\task_1613996340\appdata\local\temp\tmpti6_pi.mozrunner --skipHar --pageLoadStrategy none --webdriverPageload true --firefox.disableBrowsertimeExtension true --pageCompleteCheckStartWait 5000 --pageCompleteCheckPollTimeout 1000 --timeouts.pageLoad 60000 --timeouts.script 120000 --resultDir C:\Users\task_1613996340\build\blobber_upload_dir\browsertime-results\paypal --firefox.env MOZ_WEBRENDER=1 --firefox.env MOZ_ACCELERATED=1 --video true --visualMetrics false --firefox.windowRecorder true -n 25
[task 2021-02-24T03:10:23.282Z] 03:10:23 INFO - raptor-browsertime Info: browsertime_ffmpeg: C:/Users/task_1613996340/fetches/ffmpeg-4.1.1-win64-static/bin/ffmpeg.exe
[task 2021-02-24T03:10:23.282Z] 03:10:23 INFO - raptor-browsertime Info: PATH: C:\Users\task_1613996340\fetches\ffmpeg-4.1.1-win64-static\bin;C:\Users\task_1613996340\build\venv\Scripts;C:\windows\system32;C:\windows;C:\windows\System32\Wbem;C:\windows\System32\WindowsPowerShell\v1.0;C:\windows\System32\OpenSSH;C:\Program Files\Puppet Labs\Puppet\bin;C:\Program Files\Mellanox\MLNX_VPI\IB\Tools;C:\Program Files\Mellanox\MLNX_CIMProvider\lib\mft;C:\Program Files\Git\cmd;C:\Program Files (x86)\GNU\GnuPG\pub;C:\Program Files\Mercurial;C:\Program Files\Mercurial;C:\mozilla-build\bin;C:\mozilla-build\kdiff;C:\mozilla-build\moztools-x64\bin;C:\mozilla-build\mozmake;C:\mozilla-build\nsis-3.01;C:\mozilla-build\python;C:\mozilla-build\python\Scripts;C:\mozilla-build\python3;C:\mozilla-build\msys\bin;C:\mozilla-build\msys\local\bin;C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit;C:\Users\task_1613996340\AppData\Local\Microsoft\WindowsApps
[task 2021-02-24T03:10:24.065Z] 03:10:24 INFO - raptor-browsertime Info: Running tests using Firefox - 25 iteration(s)
[task 2021-02-24T03:11:24.069Z] 03:11:24 INFO - raptor-browsertime Info: firefox failed to start, trying 2 more time(s): Failed to start firefox in 60 seconds.
[task 2021-02-24T03:12:24.070Z] 03:12:24 INFO - raptor-browsertime Info: firefox failed to start, trying 1 more time(s): Failed to start firefox in 60 seconds.
[task 2021-02-24T03:13:24.079Z] 03:13:24 INFO - raptor-browsertime Info: firefox failed to start, trying 0 more time(s): Failed to start firefox in 60 seconds.
[task 2021-02-24T03:16:54.098Z] 03:16:54 CRITICAL - raptor-browsertime Critical: IO Completion Port failed to signal process shutdown
[task 2021-02-24T03:16:54.098Z] 03:16:54 INFO - raptor-mitmproxy Info: MitmproxyDesktop stop!!
[task 2021-02-24T03:16:54.098Z] 03:16:54 INFO - raptor-mitmproxy Info: Mitmproxy stop!!
[task 2021-02-24T03:16:54.098Z] 03:16:54 INFO - raptor-mitmproxy Info: Stopping mitmproxy playback, killing process 1216
[task 2021-02-24T03:16:54.098Z] 03:16:54 INFO - raptor-mitmproxy Info: Sending CTRL_BREAK_EVENT to mitmproxy
[task 2021-02-24T03:16:56.109Z] 03:16:56 INFO - raptor-mitmproxy Info: Successfully killed the mitmproxy playback process
[task 2021-02-24T03:16:56.109Z] 03:16:56 INFO - raptor-mitmproxy Info: Turning off the browser proxy
[task 2021-02-24T03:16:56.109Z] 03:16:56 INFO - raptor-mitmproxy Info: writing: C:\Users\task_1613996340\build\application\firefox\distribution\policies.json
[task 2021-02-24T03:16:56.114Z] 03:16:56 INFO - Exception in thread ProcessReader:
[task 2021-02-24T03:16:56.114Z] 03:16:56 ERROR - Traceback (most recent call last):
[task 2021-02-24T03:16:56.114Z] 03:16:56 INFO - File "c:\mozilla-build\python\lib\threading.py", line 801, in __bootstrap_inner
[task 2021-02-24T03:16:56.114Z] 03:16:56 INFO - self.run()
[task 2021-02-24T03:16:56.114Z] 03:16:56 INFO - File "c:\mozilla-build\python\lib\threading.py", line 754, in run
[task 2021-02-24T03:16:56.114Z] 03:16:56 INFO - self.__target(*self.__args, **self.__kwargs)
[task 2021-02-24T03:16:56.114Z] 03:16:56 INFO - File "C:\Users\task_1613996340\build\venv\lib\site-packages\mozprocess\processhandler.py", line 1265, in _read
[task 2021-02-24T03:16:56.115Z] 03:16:56 INFO - callback(line.rstrip())
[task 2021-02-24T03:16:56.115Z] 03:16:56 INFO - File "C:\Users\task_1613996340\build\venv\lib\site-packages\mozprocess\processhandler.py", line 1171, in call
[task 2021-02-24T03:16:56.115Z] 03:16:56 INFO - e(*args, **kwargs)
[task 2021-02-24T03:16:56.115Z] 03:16:56 INFO - File "C:\Users\task_1613996340\build\tests\raptor\raptor\browsertime\base.py", line 419, in _line_handler
[task 2021-02-24T03:16:56.115Z] 03:16:56 INFO - raise Exception("Browsertime failed to run")
[task 2021-02-24T03:16:56.115Z] 03:16:56 INFO - Exception: Browsertime failed to run
[task 2021-02-24T03:16:56.116Z] 03:16:56 ERROR - Traceback (most recent call last):
[task 2021-02-24T03:16:56.116Z] 03:16:56 INFO - File "C:\Users\task_1613996340\build\venv\lib\site-packages\mozprocess\processhandler.py", line 701, in _custom_wait
[task 2021-02-24T03:16:56.116Z] 03:16:56 INFO - item = self._process_events.get(timeout=timeout)
[task 2021-02-24T03:16:56.116Z] 03:16:56 INFO - File "c:\mozilla-build\python\lib\Queue.py", line 176, in get
[task 2021-02-24T03:16:56.116Z] 03:16:56 INFO - raise Empty
[task 2021-02-24T03:16:56.117Z] 03:16:56 INFO - Empty
[task 2021-02-24T03:16:56.117Z] 03:16:56 INFO - IO Completion Port unexpectedly closed

Flags: needinfo?(aryx.bugmail)

Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)

Comment 23

•

4 years ago

Greg, this surfaced after browsertime on Windows got enabled in bug 1667396.

Flags: needinfo?(aryx.bugmail) → needinfo?(gmierz2)

Regressed by: 1667396

Andreea Pavel [:apavel]

Comment 24

•

4 years ago

There are 46 total failures in the last 7 days on windows10-64-ref-hw-2017 opt
Recent failure log: https://treeherder.mozilla.org/logviewer?job_id=331288016&repo=mozilla-central&lineNumber=1316

[task 2021-02-26T04:20:55.227Z] 04:20:55 INFO - raptor-browsertime Info: Stop firefox window recorder.
[task 2021-02-26T04:20:59.131Z] 04:20:59 INFO - raptor-browsertime Info: Browsertime pageload ended.
[task 2021-02-26T04:22:16.985Z] 04:22:16 INFO - raptor-browsertime Info: https://www.bing.com/search?q=barack+obama TTFB: 13ms DOMContentLoaded: 499ms firstPaint: 294ms FCP: 291ms Load: 501ms
[task 2021-02-26T04:22:16.985Z] 04:22:16 INFO - raptor-browsertime Info: https://www.bing.com/search?q=barack+obama&browsertime_run=2 TTFB: 120ms DOMContentLoaded: 213ms firstPaint: 221ms FCP: 216ms Load: 214ms
[task 2021-02-26T04:23:16.989Z] 04:23:16 INFO - raptor-browsertime Info: firefox failed to start, trying 2 more time(s): Failed to start firefox in 60 seconds.
[task 2021-02-26T04:24:16.978Z] 04:24:16 INFO - raptor-browsertime Info: firefox failed to start, trying 1 more time(s): Failed to start firefox in 60 seconds.
[task 2021-02-26T04:25:16.983Z] 04:25:16 INFO - raptor-browsertime Info: firefox failed to start, trying 0 more time(s): Failed to start firefox in 60 seconds.
[task 2021-02-26T04:28:57.995Z] 04:28:57 CRITICAL - raptor-browsertime Critical: IO Completion Port failed to signal process shutdown
[task 2021-02-26T04:28:57.995Z] 04:28:57 INFO - raptor-mitmproxy Info: MitmproxyDesktop stop!!
[task 2021-02-26T04:28:57.995Z] 04:28:57 INFO - raptor-mitmproxy Info: Mitmproxy stop!!
[task 2021-02-26T04:28:57.996Z] 04:28:57 INFO - raptor-mitmproxy Info: Stopping mitmproxy playback, killing process 5320
[task 2021-02-26T04:28:57.996Z] 04:28:57 INFO - raptor-mitmproxy Info: Sending CTRL_BREAK_EVENT to mitmproxy
[task 2021-02-26T04:29:04.947Z] 04:29:04 INFO - raptor-mitmproxy Info: Successfully killed the mitmproxy playback process with exit code 572
[task 2021-02-26T04:29:04.947Z] 04:29:04 INFO - raptor-mitmproxy Info: Turning off the browser proxy
[task 2021-02-26T04:29:04.947Z] 04:29:04 INFO - raptor-mitmproxy Info: writing: C:\Users\task_1614306667\build\application\firefox\distribution\policies.json
[task 2021-02-26T04:29:05.271Z] 04:29:05 INFO - Exception in thread ProcessReader:
[task 2021-02-26T04:29:05.271Z] 04:29:05 ERROR - Traceback (most recent call last):
[task 2021-02-26T04:29:05.271Z] 04:29:05 INFO - File "c:\mozilla-build\python\lib\threading.py", line 801, in __bootstrap_inner
[task 2021-02-26T04:29:05.271Z] 04:29:05 INFO - self.run()
[task 2021-02-26T04:29:05.271Z] 04:29:05 INFO - File "c:\mozilla-build\python\lib\threading.py", line 754, in run
[task 2021-02-26T04:29:05.272Z] 04:29:05 INFO - self.__target(*self.__args, **self.__kwargs)
[task 2021-02-26T04:29:05.272Z] 04:29:05 INFO - File "C:\Users\task_1614306667\build\venv\lib\site-packages\mozprocess\processhandler.py", line 1272, in _read
[task 2021-02-26T04:29:05.272Z] 04:29:05 INFO - callback(line.rstrip())
[task 2021-02-26T04:29:05.272Z] 04:29:05 INFO - File "C:\Users\task_1614306667\build\venv\lib\site-packages\mozprocess\processhandler.py", line 1178, in call
[task 2021-02-26T04:29:05.272Z] 04:29:05 INFO - e(*args, **kwargs)
[task 2021-02-26T04:29:05.272Z] 04:29:05 INFO - File "C:\Users\task_1614306667\build\tests\raptor\raptor\browsertime\base.py", line 419, in _line_handler
[task 2021-02-26T04:29:05.273Z] 04:29:05 INFO - raise Exception("Browsertime failed to run")
[task 2021-02-26T04:29:05.273Z] 04:29:05 INFO - Exception: Browsertime failed to run
[task 2021-02-26T04:29:05.273Z] 04:29:05 ERROR - Traceback (most recent call last):
[task 2021-02-26T04:29:05.273Z] 04:29:05 INFO - File "C:\Users\task_1614306667\build\venv\lib\site-packages\mozprocess\processhandler.py", line 708, in _custom_wait
[task 2021-02-26T04:29:05.273Z] 04:29:05 INFO - item = self._process_events.get(timeout=timeout)
[task 2021-02-26T04:29:05.273Z] 04:29:05 INFO - File "c:\mozilla-build\python\lib\Queue.py", line 176, in get
[task 2021-02-26T04:29:05.273Z] 04:29:05 INFO - raise Empty
[task 2021-02-26T04:29:05.274Z] 04:29:05 INFO - Empty
[task 2021-02-26T04:29:05.274Z] 04:29:05 INFO - IO Completion Port unexpectedly closed
[task 2021-02-26T04:29:05.274Z] 04:29:05 ERROR - Traceback (most recent call last):
[task 2021-02-26T04:29:05.274Z] 04:29:05 INFO - File "C:\Users\task_1614306667\build\tests\raptor\raptor\raptor.py", line 212, in <module>
[task 2021-02-26T04:29:05.274Z] 04:29:05 INFO - main()
[task 2021-02-26T04:29:05.274Z] 04:29:05 INFO - File "C:\Users\task_1614306667\build\tests\raptor\raptor\raptor.py", line 158, in main
[task 2021-02-26T04:29:05.274Z] 04:29:05 INFO - success = raptor.run_tests(raptor_test_list, raptor_test_names)
[task 2021-02-26T04:29:05.274Z] 04:29:05 INFO - File "C:\Users\task_1614306667\build\tests\raptor\raptor\perftest.py", line 395, in run_tests
[task 2021-02-26T04:29:05.275Z] 04:29:05 INFO - self.run_test(test, timeout=int(test.get("page_timeout")))
[task 2021-02-26T04:29:05.275Z] 04:29:05 INFO - File "C:\Users\task_1614306667\build\tests\raptor\raptor\browsertime\base.py", line 455, in run_test
[task 2021-02-26T04:29:05.275Z] 04:29:05 INFO - proc.wait()
[task 2021-02-26T04:29:05.275Z] 04:29:05 INFO - File "C:\Users\task_1614306667\build\venv\lib\site-packages\mozprocess\processhandler.py", line 1096, in wait
[task 2021-02-26T04:29:05.275Z] 04:29:05 INFO - self.returncode = self.proc.wait()
[task 2021-02-26T04:29:05.275Z] 04:29:05 INFO - File "C:\Users\task_1614306667\build\venv\lib\site-packages\mozprocess\processhandler.py", line 284, in wait
[task 2021-02-26T04:29:05.275Z] 04:29:05 INFO - self.returncode = self._custom_wait(timeout=timeout)
[task 2021-02-26T04:29:05.275Z] 04:29:05 INFO - File "C:\Users\task_1614306667\build\venv\lib\site-packages\mozprocess\processhandler.py", line 715, in _custom_wait
[task 2021-02-26T04:29:05.276Z] 04:29:05 INFO - "IO Completion Port failed to signal process shutdown"
[task 2021-02-26T04:29:05.276Z] 04:29:05 INFO - OSError: IO Completion Port failed to signal process shutdown
[task 2021-02-26T04:29:05.464Z] 04:29:05 ERROR - Return code: 1
[task 2021-02-26T04:29:05.464Z] 04:29:05 WARNING - setting return code to 1

Whiteboard: [stockwell needswork:owner]

Comment hidden (Intermittent Failures Robot)

Greg Mierzwinski [:sparky]

Comment 26

•

4 years ago

I'm not entirely sure why but this issue seems to be resolved. There have been no failures since Feb 25th. I think the patch from bug 1694525 may have fixed it.

Flags: needinfo?(gmierz2)

See Also: → https://bugzilla.mozilla.org/show_bug.cgi?id=1694525

Henrik Skupin [:whimboo][⌚️UTC+2]

Comment 28

•

4 years ago

Related traceback here:

https://treeherder.mozilla.org/logviewer?job_id=332084116&repo=mozilla-beta&lineNumber=48610-48615

[task 2021-03-04T18:42:38.476Z] 18:42:38     INFO -  Traceback (most recent call last):
[task 2021-03-04T18:42:38.476Z] 18:42:38     INFO -    File "Z:\task_1614880213\build\venv\lib\site-packages\mozprocess\processhandler.py", line 701, in _custom_wait
[task 2021-03-04T18:42:38.476Z] 18:42:38     INFO -      item = self._process_events.get(timeout=timeout)
[task 2021-03-04T18:42:38.476Z] 18:42:38     INFO -    File "c:\mozilla-build\python\lib\Queue.py", line 176, in get
[task 2021-03-04T18:42:38.476Z] 18:42:38     INFO -      raise Empty
[task 2021-03-04T18:42:38.476Z] 18:42:38     INFO -  Empty

Status: REOPENED → NEW

OS: Unspecified → Windows

Hardware: Unspecified → All

Comment hidden (Intermittent Failures Robot)

BugBot [:suhaib / :marco/ :calixte]

Updated

•

3 years ago

Keywords: regression

Comment hidden (Intermittent Failures Robot)

BugBot [:suhaib / :marco/ :calixte]

Comment 47

•

3 years ago

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit auto_nag documentation.

Status: NEW → RESOLVED

Closed: 4 years ago → 3 years ago

Resolution: --- → INCOMPLETE

Cristina Cozmuta (:CrissCozmuta)

Comment 48

•

3 years ago

Recent failure log : https://treeherder.mozilla.org/logviewer?job_id=351039949&repo=mozilla-central&lineNumber=35296

Status: RESOLVED → REOPENED

Resolution: INCOMPLETE → ---

Comment hidden (Intermittent Failures Robot)

Alexandru Michis [:malexandru]

Updated

•

3 years ago

See Also: → https://bugzilla.mozilla.org/show_bug.cgi?id=1736256

Comment hidden (Intermittent Failures Robot)

BugBot [:suhaib / :marco/ :calixte]

Comment 59

•

3 years ago

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit auto_nag documentation.

Status: REOPENED → RESOLVED

Closed: 3 years ago → 3 years ago

Resolution: --- → INCOMPLETE

Treeherder Bug Filer

Reporter

Comment 60

•

3 years ago

treeherder

New failure instance: https://treeherder.mozilla.org/logviewer?job_id=363849497&repo=try

Status: RESOLVED → REOPENED

Resolution: INCOMPLETE → ---

Comment hidden (Intermittent Failures Robot)

BugBot [:suhaib / :marco/ :calixte]

Comment 69

•

3 years ago

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit auto_nag documentation.

Status: REOPENED → RESOLVED

Closed: 3 years ago → 3 years ago

Resolution: --- → INCOMPLETE

Treeherder Bug Filer

Reporter

Comment 70

•

3 years ago

treeherder

New failure instance: https://treeherder.mozilla.org/logviewer?job_id=375734566&repo=autoland

Status: RESOLVED → REOPENED

Resolution: INCOMPLETE → ---

Comment hidden (Intermittent Failures Robot)

BugBot [:suhaib / :marco/ :calixte]

Comment 84

•

2 years ago

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit auto_nag documentation.

Status: REOPENED → RESOLVED

Closed: 3 years ago → 2 years ago

Resolution: --- → INCOMPLETE

Treeherder Bug Filer

Reporter

Comment 85

•

2 years ago

treeherder

New failure instance: https://treeherder.mozilla.org/logviewer?job_id=390608487&repo=mozilla-central

Status: RESOLVED → REOPENED

Resolution: INCOMPLETE → ---

Comment hidden (Intermittent Failures Robot)

BMO Automation

Updated

•

2 years ago

Severity: normal → S3

Comment hidden (Intermittent Failures Robot)

Henrik Skupin [:whimboo][⌚️UTC+2]

Updated

•

2 years ago

Duplicate of this bug: 1801001

Comment hidden (Intermittent Failures Robot)

Henrik Skupin [:whimboo][⌚️UTC+2]

Updated

•

2 years ago

Duplicate of this bug: 1801749

Comment hidden (Intermittent Failures Robot)

Henrik Skupin [:whimboo][⌚️UTC+2]

Updated

•

2 years ago

Duplicate of this bug: 1803084

Comment hidden (Intermittent Failures Robot)

Henrik Skupin [:whimboo][⌚️UTC+2]

Comment 105

•

2 years ago

Hi Geoff, earlier today I have written a Marionette tests to stress-test Firefox restarts. As it looks like with an iteration of 40 we can trigger this failure permanently for CI jobs. Maybe it could help to investigate the problem we are seeing here in case it's the same underlying one? What do you think? I'm happy to trigger another try with mozprocess logging turned on as well if that would help. Just let me know.

https://treeherder.mozilla.org/jobs?repo=try&revision=1e6fd04a1dc3025ac37c69da75cc7d704741c9fb&selectedTaskRun=D32uUCFKRxCDcCU4Zyr5vA.0

Flags: needinfo?(gbrown)

Geoff Brown [:gbrown]

Comment 106

•

2 years ago

Oh, that's interesting - well done!
Unfortunately, I don't think I have time to look into it - sorry.

Flags: needinfo?(gbrown)

Comment hidden (Intermittent Failures Robot)

Henrik Skupin [:whimboo][⌚️UTC+2]

Updated

•

2 years ago

Duplicate of this bug: 1824129

Comment hidden (Intermittent Failures Robot)

Henrik Skupin [:whimboo][⌚️UTC+2]

Comment 115

•

2 years ago

Note that this failure was mainly happen on Windows. Maybe my upcoming patch on bug 1493796 might help for that platform.

Status: REOPENED → NEW

Depends on: 1493796

Comment hidden (Intermittent Failures Robot)

Henrik Skupin [:whimboo][⌚️UTC+2]

Updated

•

1 year ago

Duplicate of this bug: 1848250

Comment hidden (Intermittent Failures Robot)

BugBot [:suhaib / :marco/ :calixte]

Comment 127

•

1 year ago

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit BugBot documentation.

Status: NEW → RESOLVED

Closed: 2 years ago → 1 year ago

Resolution: --- → INCOMPLETE