toolkit/devtools/apps/tests/Makefile.in contains a reference to XPCSHELL_TESTS. Since it isn't in moz.build, these tests aren't getting executed as of the master xpcshell.ini being auto-generated from moz.build. Derp.

Trivial patch. I'm delegating review power to mshal.

gps, my question on #developers was more about other potential such disabling. But it looks like no other Makefile have XPCSHELL_TESTS in them. Regarding tookit/devtools/apps tests, they regressed while being hidden, so enabling them immediatly is going to make tests fail. I'm working on fixing those tests in bug 893848.

This patch adds code that will trigger an error if a Makefile.in contains XPCSHELL_TESTS. If only certain tests are failing, we should update the xpcshell.ini to disable them. We don't want more tests to start silently failing! And, we want the build config to be more strict.

Comment on attachment 779387 [details] [diff] [review] Re-enable accidentally disabled toolkit/devtools/apps xpcshell tests Looks good!

Can this land?

I'd love to land this! However, a test is failing locally: 0:00.37 TEST-INFO | (xpcshell/head.js) | test testCloseApp finished (2) 0:00.37 TEST-INFO | /Users/gps/src/firefox/obj-firefox.noindex/_tests/xpcshell/toolkit/devtools/apps/tests/unit/test_webappsActor.js | Starting testGetIcon 0:00.37 0:00.37 TEST-INFO | (xpcshell/head.js) | test testGetIcon pending (2) 0:00.37 0:00.37 TEST-UNEXPECTED-FAIL | /Users/gps/src/firefox/obj-firefox.noindex/_tests/xpcshell/toolkit/devtools/apps/tests/unit/test_webappsActor.js | ReferenceError: APP_ORIGIN is not defined at /Users/gps/src/firefox/obj-firefox.noindex/_tests/xpcshell/toolkit/devtools/apps/tests/unit/test_webappsActor.js:122 - see following stack: 0:00.37 testGetIcon@/Users/gps/src/firefox/obj-firefox.noindex/_tests/xpcshell/toolkit/devtools/apps/tests/unit/test_webappsActor.js:122 0:00.37 _run_next_test@/Users/gps/src/firefox/testing/xpcshell/head.js:1181 0:00.37 do_execute_soon/<.run@/Users/gps/src/firefox/testing/xpcshell/head.js:444 0:00.37 0:00.37 0:00.37 TEST-INFO | (xpcshell/head.js) | exiting test 0:00.37 0:00.37 TEST-INFO | (xpcshell/head.js) | test run_next_test 6 finished (2) 0:00.37 0:00.37 TEST-PASS | /Users/gps/src/firefox/obj-firefox.noindex/_tests/xpcshell/toolkit/devtools/apps/tests/unit/test_webappsActor.js | [testCloseApp/< : 111] false == false 0:00.37 0:00.37 TEST-INFO | (xpcshell/head.js) | test run_next_test 7 pending (2) 0:00.37 0:00.37 TEST-INFO | (xpcshell/head.js) | test testGetIcon finished (2) 0:00.37 TEST-INFO | /Users/gps/src/firefox/obj-firefox.noindex/_tests/xpcshell/toolkit/devtools/apps/tests/unit/test_webappsActor.js | Starting testGetIconWithCustomSize 0:00.37 0:00.37 TEST-INFO | (xpcshell/head.js) | test testGetIconWithCustomSize pending (2) 0:00.37 0:00.37 TEST-UNEXPECTED-FAIL | /Users/gps/src/firefox/obj-firefox.noindex/_tests/xpcshell/toolkit/devtools/apps/tests/unit/test_webappsActor.js | ReferenceError: APP_ORIGIN is not defined at /Users/gps/src/firefox/obj-firefox.noindex/_tests/xpcshell/toolkit/devtools/apps/tests/unit/test_webappsActor.js:138 - see following stack: 0:00.37 testGetIconWithCustomSize@/Users/gps/src/firefox/obj-firefox.noindex/_tests/xpcshell/toolkit/devtools/apps/tests/unit/test_webappsActor.js:138 0:00.37 _run_next_test@/Users/gps/src/firefox/testing/xpcshell/head.js:1181 0:00.37 do_execute_soon/<.run@/Users/gps/src/firefox/testing/xpcshell/head.js:444 0:00.37 0:00.37 0:00.37 TEST-INFO | (xpcshell/head.js) | exiting test 0:00.37 0:00.37 TEST-INFO | (xpcshell/head.js) | test run_next_test 7 finished (2) 0:00.37 <<<<<<< 0:00.38 TEST-INFO | /Users/gps/src/firefox/obj-firefox.noindex/_tests/xpcshell/toolkit/devtools/apps/tests/unit/test_appInstall.js | running test ... 0:00.65 TEST-PASS | /Users/gps/src/firefox/obj-firefox.noindex/_tests/xpcshell/toolkit/devtools/apps/tests/unit/test_appInstall.js | test passed (time: 272.390ms) 0:00.66 INFO | Result summary: 0:00.66 INFO | Passed: 1 0:00.66 INFO | Failed: 1 0:00.66 INFO | Todo: 0

Let's disable just the one test per comment 3, then.

I'm still unable to make them to work permanently. So yes, please disable these tests explicitely in the meantime.

https://hg.mozilla.org/integration/fx-team/rev/71fa40e0ced3 With both tests disabled.