No such file or directory: 'taskcluster/ci/test/variants.yml' during bootstrap
Categories
(Firefox Build System :: General, defect)
Tracking
(firefox-esr91 unaffected, firefox93 unaffected, firefox94 unaffected, firefox95 unaffected, firefox96+ fixed)
Tracking | Status | |
---|---|---|
firefox-esr91 | --- | unaffected |
firefox93 | --- | unaffected |
firefox94 | --- | unaffected |
firefox95 | --- | unaffected |
firefox96 | + | fixed |
People
(Reporter: yury, Assigned: ahal)
References
(Blocks 1 open bug, Regression)
Details
(Keywords: regression)
While running ./mach bootstrap
for "2. Firefox for Desktop" I'm getting:
The details of the failure are as follows:
FileNotFoundError: [Errno 2] No such file or directory: 'taskcluster/ci/test/variants.yml'
File "/Users/yury/Work/mozilla-unified/python/mozbuild/mozbuild/artifact_commands.py", line 373, in artifact_toolchain
tasks = toolchain_task_definitions()
File "/Users/yury/Work/mozilla-unified/python/mozbuild/mozbuild/toolchains.py", line 19, in toolchain_task_definitions
toolchains = load_tasks_for_kind(params, "toolchain", root_dir=root_dir)
File "/Users/yury/Work/mozilla-unified/taskcluster/gecko_taskgraph/generator.py", line 489, in load_tasks_for_kind
for task in tgg.full_task_set
File "/Users/yury/Work/mozilla-unified/taskcluster/gecko_taskgraph/generator.py", line 162, in full_task_set
return self._run_until("full_task_set")
File "/Users/yury/Work/mozilla-unified/taskcluster/gecko_taskgraph/generator.py", line 434, in _run_until
k, v = next(self._run)
File "/Users/yury/Work/mozilla-unified/taskcluster/gecko_taskgraph/generator.py", line 310, in _run
new_tasks = kind.load_tasks(
File "/Users/yury/Work/mozilla-unified/taskcluster/gecko_taskgraph/generator.py", line 77, in load_tasks
tasks = [
File "/Users/yury/Work/mozilla-unified/taskcluster/gecko_taskgraph/generator.py", line 77, in <listcomp>
tasks = [
File "/Users/yury/Work/mozilla-unified/taskcluster/gecko_taskgraph/transforms/task.py", line 2093, in check_run_task_caches
for task in tasks:
File "/Users/yury/Work/mozilla-unified/taskcluster/gecko_taskgraph/transforms/task.py", line 2003, in check_task_dependencies
for task in tasks:
File "/Users/yury/Work/mozilla-unified/taskcluster/gecko_taskgraph/transforms/task.py", line 1989, in check_task_identifiers
for task in tasks:
File "/Users/yury/Work/mozilla-unified/taskcluster/gecko_taskgraph/transforms/task.py", line 1970, in chain_of_trust
for task in tasks:
File "/Users/yury/Work/mozilla-unified/taskcluster/gecko_taskgraph/transforms/task.py", line 1761, in build_task
for task in tasks:
File "/Users/yury/Work/mozilla-unified/taskcluster/gecko_taskgraph/transforms/task.py", line 1752, in try_task_config_routes
for task in tasks:
File "/Users/yury/Work/mozilla-unified/taskcluster/gecko_taskgraph/transforms/task.py", line 1742, in try_task_config_chemspill_prio
for task in tasks:
File "/Users/yury/Work/mozilla-unified/taskcluster/gecko_taskgraph/transforms/task.py", line 1732, in try_task_config_env
for task in tasks:
File "/Users/yury/Work/mozilla-unified/taskcluster/gecko_taskgraph/transforms/task.py", line 1694, in add_index_routes
for task in tasks:
File "/Users/yury/Work/mozilla-unified/taskcluster/gecko_taskgraph/transforms/task.py", line 1489, in validate
for task in tasks:
File "/Users/yury/Work/mozilla-unified/taskcluster/gecko_taskgraph/transforms/task.py", line 1466, in task_name_from_label
for task in tasks:
File "/Users/yury/Work/mozilla-unified/taskcluster/gecko_taskgraph/transforms/task.py", line 1421, in set_defaults
for task in tasks:
File "/Users/yury/Work/mozilla-unified/taskcluster/gecko_taskgraph/transforms/task.py", line 1400, in set_implementation
for task in tasks:
File "/Users/yury/Work/mozilla-unified/taskcluster/gecko_taskgraph/transforms/job/init.py", line 384, in make_task_description
import_sibling_modules(exceptions=("common.py",))
File "/Users/yury/Work/mozilla-unified/taskcluster/gecko_taskgraph/util/python_path.py", line 53, in import_sibling_modules
import(modpath + "." + f[:-3])
File "/Users/yury/Work/mozilla-unified/taskcluster/gecko_taskgraph/transforms/job/mozharness_test.py", line 19, in <module>
from gecko_taskgraph.transforms.tests import test_description_schema, normpath
File "/Users/yury/Work/mozilla-unified/taskcluster/gecko_taskgraph/transforms/tests.py", line 216, in <module>
TEST_VARIANTS = load_yaml("taskcluster", "ci", "test", "variants.yml")
File "/Users/yury/Work/mozilla-unified/third_party/python/taskcluster_taskgraph/taskgraph/util/yaml.py", line 35, in load_yaml
with open(filename, "rb") as f:
Error running mach:
['bootstrap']
Comment 1•3 years ago
|
||
File "/Users/yury/Work/mozilla-unified/taskcluster/gecko_taskgraph/transforms/tests.py", line 216, in <module>
TEST_VARIANTS = load_yaml("taskcluster", "ci", "test", "variants.yml")
This suggests you are on a checkout of autoland, since this landed in bug 1737673, and the taskcluster/ci/test/variants.yml file was added at the same time, so it seems your checkout is broken.
Comment 2•3 years ago
|
||
I did a pull/rebase on bookmarks/central, and started seeing this. Definitely not on autoland, just the latest mozilla-central.
https://hg.mozilla.org/mozilla-central/rev/7e8e3747c3f81e844e17ae8d645dd6091bafcae7
The file in question exists at the quoted location.
(Andrew beat me to it, but here's my way-too-long comment anyway, in case it also helps.)
I got the same error after updating to the latest central https://hg.mozilla.org/mozilla-central/rev/7e8e3747c3f81e844e17ae8d645dd6091bafcae7
Starting from the previous (still working) central https://hg.mozilla.org/mozilla-central/rev/1f57920fc04062c989c749169a31d147227a2962 ,
I found that the first patch to cause trouble is https://hg.mozilla.org/mozilla-central/rev/e38b571d248d345c8a501065c7cc853da3dfcfd1 : Bug 1737673 - Pull 'variant' definitions into a yaml file, r=jmaher
Indeed, it's where this apparently-missing file is added.
The error is the same for me, but with the equivalent path style for Windows:
FileNotFoundError: [Errno 2] No such file or directory: 'taskcluster\\ci\\test\\variants.yml'
I can see that the file is really there, exactly taskcluster\ci\test\variants.yml under my repository root.
Is the check done from another directory, maybe?
And I guess Yury may have seen this in Autoland after that patch landed there; but now that patch has been merged into central, so others will hit this issue.
Updated•3 years ago
|
Comment 4•3 years ago
|
||
It appears the current working directory is /home/aosmond/.mozbuild
instead of /home/aosmond/dev/gecko
. Hacking it to use an absolute path allowed me to finish the bootstrap and begin building:
diff --git a/taskcluster/gecko_taskgraph/transforms/tests.py b/taskcluster/gecko_taskgraph/transforms/tests.py
index 1605bfb77ffd0..b7eaf504efafb 100644
--- a/taskcluster/gecko_taskgraph/transforms/tests.py
+++ b/taskcluster/gecko_taskgraph/transforms/tests.py
@@ -213,7 +213,7 @@ MACOSX_WORKER_TYPES = {
}
-TEST_VARIANTS = load_yaml("taskcluster", "ci", "test", "variants.yml")
+TEST_VARIANTS = load_yaml("/home/aosmond/dev/gecko/taskcluster/ci/test/variants.yml")
"""List of available test variants defined."""
DYNAMIC_CHUNK_DURATION = 20 * 60 # seconds
Updated•3 years ago
|
Updated•3 years ago
|
Updated•3 years ago
|
Comment 6•3 years ago
|
||
Fixed by backout. I can confirm that local builds are working again off m-c tip.
https://hg.mozilla.org/mozilla-central/rev/47f2f2add0cae4cb58825876f635b39d4dfc4839
Description
•