eg, after a successful build: """ $ ./mach xpcshell-test services/sync/tests/unit/ From _tests: Kept 14480 existing; Added/updated 1; Removed 0 files and 0 directories. 0:06.15 Found node at c:\src\mozilla-git\gecko-dev\testing\xpcshell\node 0:06.15 Found moz-spdy at c:\src\mozilla-git\gecko-dev\testing\xpcshell\moz-spdy\moz-spdy.js 0:06.15 Could not run moz-spdy server: [Error 193] %1 is not a valid Win32 application 0:06.15 Found moz-http2 at c:\src\mozilla-git\gecko-dev\testing\xpcshell\moz-http2\moz-http2.js 0:06.15 Could not run moz-http2 server: [Error 193] %1 is not a valid Win32 application 0:06.21 INFO | Using at most 16 threads. 0:06.21 TEST-INFO | test_addons_reconciler.js | Test failed or timed out, will retry. <snip lots of test names with the same error message> 0:06.27 Retrying tests that failed when run in parallel. 0:06.27 INFO | Following exceptions were raised: 0:06.27 Traceback (most recent call last): File "c:\src\mozilla-git\gecko-dev\testing/xpcshell\runxpcshelltests.py", line 145, in run self.run_test() File "c:\src\mozilla-git\gecko-dev\testing/xpcshell\runxpcshelltests.py", line 537, in run_test self.test_object['here']) Exception: tests_root_dir is not a parent path of c:/src/mozilla-git/gecko-dev/obj-release/_tests/xpcshell/services/sync/tests/unit 0:06.27 Traceback (most recent call last): File "c:\src\mozilla-git\gecko-dev\testing/xpcshell\runxpcshelltests.py", line 145, in run self.run_test() File "c:\src\mozilla-git\gecko-dev\testing/xpcshell\runxpcshelltests.py", line 537, in run_test self.test_object['here']) Exception: tests_root_dir is not a parent path of c:/src/mozilla-git/gecko-dev/obj-release/_tests/xpcshell/services/sync/tests/unit Error running mach: ['xpcshell-test', 'services/sync/tests/unit/'] The error occurred in code that was called by the mach command. This is either a bug in the called code itself or in the way that mach is calling it. You should consider filing a bug for this issue. If filing a bug, please include the full output of mach, including this error message. The details of the failure are as follows: Exception: tests_root_dir is not a parent path of c:/src/mozilla-git/gecko-dev/obj-release/_tests/xpcshell/services/sync/tests/unit File "c:\src\mozilla-git\gecko-dev\testing/xpcshell/mach_commands.py", line 339, in run_xpcshell_test return xpcshell.run_test(**params) File "c:\src\mozilla-git\gecko-dev\testing/xpcshell/mach_commands.py", line 102, in run_test return self._run_xpcshell_harness(**args) File "c:\src\mozilla-git\gecko-dev\testing/xpcshell/mach_commands.py", line 187, in _run_xpcshell_harness result = xpcshell.runTests(**filtered_args) File "c:\src\mozilla-git\gecko-dev\testing/xpcshell\runxpcshelltests.py", line 1483, in runTests raise exceptions[0] /c/src/mozilla-git/gecko-dev - .mozconfig-release - (fxa-sync) $ """ Trying to specify a single file in that directory produces the same basic error. Trying to specify a manifest file: """ $ ./mach xpcshell-test services/sync/tests/unit/xpcshell.ini From _tests: Kept 14480 existing; Added/updated 1; Removed 0 files and 0 directories. We could not find an xpcshell test for the passed test path. Please select a path that is a test file or is a directory containing xpcshell tests. """ Running 'xpcshell-tests' with no arguments works as expected. I've tried this with 2 different xpcshell test directories. environment: $ ./mach environment platform: Windows-8-6.2.9200 python version: 2.7.4 (default, Apr 6 2013, 19:54:46) [MSC v.1500 32 bit (Intel)] python prefix: c:\mozilla-build\python mach cwd: c:\src\mozilla-git\gecko-dev os cwd: c:\src\mozilla-git\gecko-dev mach directory: c:\src\mozilla-git\gecko-dev state directory: c:/home/.mozbuild mozconfig path: c:\src\mozilla-git\gecko-dev\.mozconfig-release object directory: c:\src\mozilla-git\gecko-dev\obj-release mozconfig configure args: --enable-application=browser --enable-tests --enable-jemalloc --enable-signmar --enable-profiling --enable-js-diagnostics mozconfig extra make args: mozconfig make flags: - j 6 config topsrcdir: c:/src/mozilla-git/gecko-dev config topobjdir: c:/src/mozilla-git/gecko-dev/obj-release

It turns out that line 534: self.tests_root_dir = os.path.normpath(self.tests_root_dir) causes self.tests_root_dir to have backslashes, while self.test_object['here'] has forward slashes. Using .normpath on the test path seems to solve the problem (as, presumably, would using posix.normpath on the root dir)

Comment on attachment 8365723 [details] [diff] [review] 0011-Bug-964102-use-normpath-on-the-test-path-before-comp.patch Review of attachment 8365723 [details] [diff] [review]: ----------------------------------------------------------------- ::: testing/xpcshell/runxpcshelltests.py @@ +531,5 @@ > # dir and the test with path characters replaced with '.' (using Java > # class notation). > if self.tests_root_dir is not None: > self.tests_root_dir = os.path.normpath(self.tests_root_dir) > + if os.path.normpath(self.test_object['here']).find(self.tests_root_dir) != 0: You are on the right track. We should use mozpack.mozpath.normpath for all path comparisons. (That's effectively posixpath.normpath.)

Comment on attachment 8365723 [details] [diff] [review] 0011-Bug-964102-use-normpath-on-the-test-path-before-comp.patch Review of attachment 8365723 [details] [diff] [review]: ----------------------------------------------------------------- Ignore my comment. all-tests.json doesn't store normalized paths. This is definitely a follow-up waiting to be filed. This is the correct solution given the current implementation.

I'd land this, but the trees are closed. Since this is developer impacting, we may want to land in all of the trees, or at least get things merged quickly.