Closed Bug 1609623 Opened 5 years ago Closed 2 years ago

Intermittent [hazard] subprocess.CalledProcessError: Command 'analyzeRoots.js' returned non-zero exit status 9

Categories

(Firefox Build System :: General, defect, P5)

defect

Tracking

(Not tracked)

RESOLVED INCOMPLETE

People

(Reporter: intermittent-bug-filer, Unassigned)

References

Details

(Keywords: intermittent-failure)

Filed by: apavel [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=285150900&repo=autoland
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/PxdHaunXSASnQHw18iYZSA/runs/0/artifacts/public/logs/live_backing.log


[task 2020-01-16T06:42:26.072Z] 68:39.08 Notification center failed: Install notify-send (usually part of the libnotify package) to get a notification when the build finishes.
[task 2020-01-16T06:42:26.076Z] To view resource usage of the build, run |mach resource-usage|.
[task 2020-01-16T06:42:26.076Z] 68:39.08 We know it took a while, but your build finally finished successfully!
[task 2020-01-16T06:42:26.077Z] To take your build for a test drive, run: |mach run|
[task 2020-01-16T06:42:26.077Z] For more information on what to do now, see https://developer.mozilla.org/docs/Developer_Guide/So_You_Just_Built_Firefox
[task 2020-01-16T06:42:26.208Z] Received connection.
[task 2020-01-16T06:42:26.209Z] Final transaction received, finishing...
[task 2020-01-16T06:42:53.021Z] Waiting for manager to finish (build status 0)...
[task 2020-01-16T06:42:53.021Z] Exiting with status 0
[task 2020-01-16T06:42:53.022Z] build finished (status 0): Thu Jan 16 06:42:53 2020
[task 2020-01-16T06:42:53.022Z] Exiting run_complete with status 0
[task 2020-01-16T06:42:53.024Z] Running gcTypes to generate ('gcTypes.txt', 'typeInfo.txt')
[task 2020-01-16T06:42:53.024Z] ANALYZED_OBJDIR='/builds/worker/workspace/obj-analyzed' SOURCE='/builds/worker/checkouts/gecko' LD_LIBRARY_PATH="${LD_LIBRARY_PATH}:/builds/worker/workspace/obj-haz-shell/dist/bin" XDB='/builds/worker/fetches/sixgill/usr/bin/xdb.so' PATH="/builds/worker/fetches/sixgill/usr/bin:${PATH}" /builds/worker/workspace/obj-haz-shell/dist/bin/js /builds/worker/checkouts/gecko/js/src/devtools/rootAnalysis/computeGCTypes.js gcTypes.txt typeInfo.txt
[task 2020-01-16T06:43:23.546Z] Running callgraph to generate ('callgraph.txt',)
[task 2020-01-16T06:43:23.546Z] ANALYZED_OBJDIR='/builds/worker/workspace/obj-analyzed' SOURCE='/builds/worker/checkouts/gecko' LD_LIBRARY_PATH="${LD_LIBRARY_PATH}:/builds/worker/workspace/obj-haz-shell/dist/bin" XDB='/builds/worker/fetches/sixgill/usr/bin/xdb.so' PATH="/builds/worker/fetches/sixgill/usr/bin:${PATH}" /builds/worker/workspace/obj-haz-shell/dist/bin/js /builds/worker/checkouts/gecko/js/src/devtools/rootAnalysis/computeCallgraph.js typeInfo.txt callgraph.txt
[task 2020-01-16T06:43:54.859Z] Finished loading data structures
[task 2020-01-16T07:06:19.586Z] Running gcFunctions to generate ('gcFunctions.txt', 'gcFunctions.lst', 'gcEdges.txt', 'limitedFunctions.lst')
[task 2020-01-16T07:06:19.587Z] ANALYZED_OBJDIR='/builds/worker/workspace/obj-analyzed' SOURCE='/builds/worker/checkouts/gecko' LD_LIBRARY_PATH="${LD_LIBRARY_PATH}:/builds/worker/workspace/obj-haz-shell/dist/bin" XDB='/builds/worker/fetches/sixgill/usr/bin/xdb.so' PATH="/builds/worker/fetches/sixgill/usr/bin:${PATH}" /builds/worker/workspace/obj-haz-shell/dist/bin/js /builds/worker/checkouts/gecko/js/src/devtools/rootAnalysis/computeGCFunctions.js callgraph.txt gcFunctions.txt gcFunctions.lst gcEdges.txt limitedFunctions.lst
[task 2020-01-16T07:07:52.462Z] Writing gcFunctions.tmp
[task 2020-01-16T07:08:08.297Z] Writing gcFunctions_list.tmp
[task 2020-01-16T07:08:12.458Z] Writing gcEdges.tmp
[task 2020-01-16T07:08:12.458Z] Writing limitedFunctions_list.tmp
[task 2020-01-16T07:08:14.662Z] Running allFunctions to generate allFunctions.txt

[task 2020-01-16T07:10:45.239Z] ANALYZED_OBJDIR='/builds/worker/workspace/obj-analyzed' SOURCE='/builds/worker/checkouts/gecko' LD_LIBRARY_PATH="${LD_LIBRARY_PATH}:/builds/worker/workspace/obj-haz-shell/dist/bin" XDB='/builds/worker/fetches/sixgill/usr/bin/xdb.so' PATH="/builds/worker/fetches/sixgill/usr/bin:${PATH}" /builds/worker/workspace/obj-haz-shell/dist/bin/js /builds/worker/checkouts/gecko/js/src/devtools/rootAnalysis/analyzeRoots.js gcFunctions.lst gcEdges.txt limitedFunctions.lst gcTypes.txt typeInfo.txt 31 31 tmp.31 > rootingHazards.31
[task 2020-01-16T07:10:45.240Z] Traceback (most recent call last):
[task 2020-01-16T07:10:45.240Z] File "/builds/worker/checkouts/gecko/js/src/devtools/rootAnalysis/analyze.py", line 339, in <module>
[task 2020-01-16T07:10:45.240Z] run_job(step, data)
[task 2020-01-16T07:10:45.240Z] File "/builds/worker/checkouts/gecko/js/src/devtools/rootAnalysis/analyze.py", line 181, in run_job
[task 2020-01-16T07:10:45.240Z] cmdspec(config, outfiles)
[task 2020-01-16T07:10:45.240Z] File "/builds/worker/checkouts/gecko/js/src/devtools/rootAnalysis/analyze.py", line 110, in generate_hazards
[task 2020-01-16T07:10:45.240Z] raise subprocess.CalledProcessError(final_status, 'analyzeRoots.js')
[task 2020-01-16T07:10:45.240Z] subprocess.CalledProcessError: Command 'analyzeRoots.js' returned non-zero exit status 9
[task 2020-01-16T07:10:45.252Z] + onexit
[task 2020-01-16T07:10:45.252Z] + grab_artifacts /builds/worker/workspace/analysis /builds/worker/artifacts
[task 2020-01-16T07:10:45.252Z] + local analysis_dir
[task 2020-01-16T07:10:45.253Z] + analysis_dir=/builds/worker/workspace/analysis
[task 2020-01-16T07:10:45.253Z] + local artifacts
[task 2020-01-16T07:10:45.253Z] + artifacts=/builds/worker/artifacts
[task 2020-01-16T07:10:45.253Z] + cd /builds/worker/workspace/analysis
[task 2020-01-16T07:10:45.253Z] + ls -lah
[task 2020-01-16T07:10:45.268Z] total 6.1G
[task 2020-01-16T07:10:45.268Z] drwxr-xr-x 3 worker worker 4.0K Jan 16 07:08 .
[task 2020-01-16T07:10:45.268Z] drwxr-xr-x 6 worker worker 4.0K Jan 16 05:33 ..

Reviewing orange factor, it looks like it follows bug 1597154 a lot more closely than it does mine. It started with it and stopped when it got backed out.

Blocks: 1597154
Flags: needinfo?(aosmond) → needinfo?(pbone)

pbone said he will take a look.

Tentatively tracking for Fission Dogfooding (M5).

Fission Milestone: --- → M5
Summary: Intermittent [hazzard] subprocess.CalledProcessError: Command 'analyzeRoots.js' returned non-zero exit status 9 → Intermittent [hazard] subprocess.CalledProcessError: Command 'analyzeRoots.js' returned non-zero exit status 9

(In reply to Andreea Pavel [:apavel] from comment #6)

Root cause of this bug and Bug 1614852 has been reverted here: https://bugzilla.mozilla.org/show_bug.cgi?id=1614852#c12
https://hg.mozilla.org/integration/autoland/rev/7dcbf3debe6a3a527df82c422446ca42a3de78c4

Since GCP bug 1614852 has reverted the root cause, can close this bug as fixed? Or is there additional work needed to verify hazard analysis is working with Fission on GCP?

The plan to switch remains, getting Hazard ready before the next switch would be welcome.

Kendall, please let Chris know about the next date for the AWS to GCP switch once it is scheduled.

For testing it before, the worker type will need to be added to the task config like this: https://searchfox.org/mozilla-central/rev/a4be2fbe9bd4f405c91cc16e4e3a80400f5a9301/taskcluster/ci/build/android.yml#384

Flags: needinfo?(klibby)

Looks like others found a root cause before I had time to look at this bug. Removing my NI.

Flags: needinfo?(pbone)

Not a Fission bug

Fission Milestone: M5 → ---

Clear a needinfo that is pending on an inactive user.

Inactive users most likely will not respond; if the missing information is essential and cannot be collected another way, the bug maybe should be closed as INCOMPLETE.

For more information, please visit auto_nag documentation.

Flags: needinfo?(klibby)
Status: NEW → RESOLVED
Closed: 2 years ago
Resolution: --- → INCOMPLETE
You need to log in before you can comment on or make changes to this bug.