I've been able to run xpcshell tests from a tup build just fine, but the mochitests fail due to: "invalid setup: missing mochikit extension"

Aha, there's a stray libs rule that copies mochijar from dist/xpi-stage to _tests/testing/mochitest: https://searchfox.org/mozilla-central/rev/da499aac682d0bbda5829327b60a865cbc491611/testing/mochitest/Makefile.in#11

I wonder if we could just fix the default path in runtests.py to look in dist/xpi-stage, and fix the test packaging bits to copy it out of there into the current place in the test package?

(In reply to Ted Mielczarek [:ted.mielczarek] from comment #2) > I wonder if we could just fix the default path in runtests.py to look in > dist/xpi-stage, and fix the test packaging bits to copy it out of there into > the current place in the test package? +1. I think the main reason we put everything in those staging test dirs is because it was previously a PITA to create the test package in testsuite-targets.mk. But now with test_archive.py, it's trivial to pull in things from all over the sourcedir/objdir.

Comment on attachment 8980455 [details] Bug 1462489 - Use test addons from dist/xpi-stage when running mochitests locally. https://reviewboard.mozilla.org/r/246618/#review252926 Lgtm!

Comment on attachment 8980454 [details] Bug 1462489 - Process some chrome manifest entries from dist/xpi-stage from the tup backend. https://reviewboard.mozilla.org/r/246616/#review254188

Comment on attachment 8980456 [details] Bug 1462489 - Put packaged test addons in the test package with the test packager rather than Makefile.in https://reviewboard.mozilla.org/r/246620/#review254190 ::: python/mozbuild/mozbuild/action/test_archive.py:372 (Diff revision 1) > + 'source': buildconfig.topobjdir, > + 'base': 'dist/xpi-stage', > + 'pattern': 'mochijar/**', > + 'dest': 'mochitest' > + }, > + { Is this entry intended to be something else? It's a duplicate of the one above.

Comment on attachment 8980457 [details] Bug 1462489 - Re-generate the build backend when a build backend output is missing. https://reviewboard.mozilla.org/r/246622/#review254194

Comment on attachment 8980456 [details] Bug 1462489 - Put packaged test addons in the test package with the test packager rather than Makefile.in https://reviewboard.mozilla.org/r/246620/#review254190 > Is this entry intended to be something else? It's a duplicate of the one above. Nope, it's just duplicated for no reason. Thanks for noticing that!

Pushed by cmanchester@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/dfed23857cea Process some chrome manifest entries from dist/xpi-stage from the tup backend. r=mshal https://hg.mozilla.org/integration/autoland/rev/3cd6a8ee8f66 Use test addons from dist/xpi-stage when running mochitests locally. r=ahal https://hg.mozilla.org/integration/autoland/rev/daa33716b140 Put packaged test addons in the test package with the test packager rather than Makefile.in r=mshal https://hg.mozilla.org/integration/autoland/rev/41cf0d53a95c Re-generate the build backend when a build backend output is missing. r=mshal

Backed out 4 changesets (bug 1462489) for breaking mochitest on testing/mochitest/tests/python/test_basic_mochitest_plain.py Log: https://treeherder.mozilla.org/logviewer.html#?job_id=181011268&repo=autoland&lineNumber=355 0:11.62 ===================== 1 passed, 1 warnings in 0.28 seconds ===================== [task 2018-05-30T23:11:04.028Z] 0:11.72 /builds/worker/checkouts/gecko/testing/mochitest/tests/python/test_basic_mochitest_plain.py [task 2018-05-30T23:11:04.639Z] 0:12.33 ============================= test session starts ============================== [task 2018-05-30T23:11:04.639Z] 0:12.33 platform linux2 -- Python 2.7.12, pytest-3.2.5, py-1.5.3, pluggy-0.4.0 -- /builds/worker/checkouts/gecko/obj-x86_64-pc-linux-gnu/_virtualenvs/init/bin/python [task 2018-05-30T23:11:04.639Z] 0:12.33 rootdir: /builds/worker/checkouts/gecko, inifile: [task 2018-05-30T23:11:04.667Z] 0:12.36 collecting ... collected 6 items [task 2018-05-30T23:11:04.668Z] 0:12.36 [task 2018-05-30T23:11:13.553Z] 0:21.24 testing/mochitest/tests/python/test_basic_mochitest_plain.py::test_output_pass TEST-UNEXPECTED-FAIL [task 2018-05-30T23:11:21.572Z] 0:29.26 testing/mochitest/tests/python/test_basic_mochitest_plain.py::test_output_fail TEST-UNEXPECTED-FAIL [task 2018-05-30T23:11:21.573Z] 0:29.26 testing/mochitest/tests/python/test_basic_mochitest_plain.py::test_output_crash SKIPPED [task 2018-05-30T23:11:31.213Z] 0:38.90 testing/mochitest/tests/python/test_basic_mochitest_plain.py::test_output_asan TEST-UNEXPECTED-FAIL [task 2018-05-30T23:11:31.213Z] 0:38.90 testing/mochitest/tests/python/test_basic_mochitest_plain.py::test_output_assertion SKIPPED [task 2018-05-30T23:11:31.216Z] 0:38.91 testing/mochitest/tests/python/test_basic_mochitest_plain.py::test_output_leak SKIPPED [task 2018-05-30T23:11:31.217Z] 0:38.91 [task 2018-05-30T23:11:31.217Z] 0:38.91 =================================== FAILURES =================================== [task 2018-05-30T23:11:31.217Z] 0:38.91 _______________________________ test_output_pass _______________________________ [task 2018-05-30T23:11:31.218Z] 0:38.91 [task 2018-05-30T23:11:31.218Z] 0:38.91 runtests = <function inner at 0x7f7e2e05c500> Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=41cf0d53a95cc8966659a5eafb93d3826b97ea62&filter-resultStatus=testfailed&filter-resultStatus=busted&filter-resultStatus=exception&filter-classifiedState=unclassified Backout: https://hg.mozilla.org/integration/autoland/rev/aca4b42750423341b74ba8cfbd77d27069f70fb2

I would have fixed it this anyway, so it's not really much of an inconvenience, but these tests are marked as tier-2, so as far as I know this failure should not have resulted in a backout.

Comment on attachment 8980455 [details] Bug 1462489 - Use test addons from dist/xpi-stage when running mochitests locally. https://reviewboard.mozilla.org/r/246618/#review254392

Pushed by cmanchester@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/7f3e1d87baa3 Process some chrome manifest entries from dist/xpi-stage from the tup backend. r=mshal https://hg.mozilla.org/integration/autoland/rev/5133f89842f6 Use test addons from dist/xpi-stage when running mochitests locally. r=ahal https://hg.mozilla.org/integration/autoland/rev/ce2105ffae17 Put packaged test addons in the test package with the test packager rather than Makefile.in r=mshal https://hg.mozilla.org/integration/autoland/rev/e0b837323b48 Re-generate the build backend when a build backend output is missing. r=mshal

https://hg.mozilla.org/mozilla-central/rev/7f3e1d87baa3 https://hg.mozilla.org/mozilla-central/rev/5133f89842f6 https://hg.mozilla.org/mozilla-central/rev/ce2105ffae17 https://hg.mozilla.org/mozilla-central/rev/e0b837323b48