Open Bug 1641900 Opened 5 years ago Updated 5 years ago

TV job on Android/GeckoView skips all extension tests, possibly because the test is registered twice

Categories

(Testing :: General, defect, P3)

defect

Tracking

(Not tracked)

People

(Reporter: robwu, Unassigned)

References

(Blocks 1 open bug)

Details

I modified some xpcshell tests in bug 1638007 and added a mochitest in bug 1641269, and then pushed to try with ./mach try auto (I also tried the job selector in the Treeherder UI). I did observe that the mochitest ran, but somehow the TV test skips all tests:

[task 2020-05-28T21:09:21.987Z] 21:09:21     INFO - 'toolkit/components/extensions/test/xpcshell/test_webRequest_cookies.js' has been skipped on this platform.
[task 2020-05-28T21:09:21.988Z] 21:09:21     INFO - Per-test run found test toolkit/content/tests/browser/browser_default_image_filename_redirect.js (mochitest-browser-chrome/None)
[task 2020-05-28T21:09:21.988Z] 21:09:21     INFO - 'toolkit/components/extensions/test/mochitest/test_ext_webrequest_and_proxy_filter.html' has been skipped on this platform.
[task 2020-05-28T21:09:21.988Z] 21:09:21     INFO - 'toolkit/components/extensions/test/xpcshell/test_webRequest_ancestors.js' has been skipped on this platform.
[task 2020-05-28T21:09:21.988Z] 21:09:21     INFO - 'toolkit/components/extensions/test/xpcshell/test_webRequest_filtering.js' has been skipped on this platform.

Possibly relevant is that these tests are registered twice and included through another test manifest.

e.g. xpcshell, tests are in xpcshell-common.ini and included:

e.g. mochitest, tests are in mochitest-common.ini and included:

I think these cases are running into the limitation noted in the comment block here:

https://searchfox.org/mozilla-central/rev/8827278483c337667cdfb238112eb1be397dd102/testing/mozharness/mozharness/mozilla/testing/per_test_base.py#56-66

If a test is disabled in one manifest and enabled in another, TV considers the test to be disabled. It should be possible to disambiguate with path+manifest.

Severity: -- → S3
Priority: -- → P3
You need to log in before you can comment on or make changes to this bug.