#[markdown(off)]
Filed by: nbeleuzu [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=228809744&repo=autoland

https://queue.taskcluster.net/v1/task/GyrEsrm9S_aqrBtg014QPg/runs/0/artifacts/public/logs/live_backing.log

[task 2019-02-16T10:59:17.560Z] 10:59:17 INFO - REFTEST SUITE-START | Running 0 tests
[task 2019-02-16T10:59:17.561Z] 10:59:17 INFO - REFTEST INFO | Result summary:
[task 2019-02-16T10:59:17.561Z] 10:59:17 INFO - REFTEST INFO | Successful: 0 (0 pass, 0 load only)
[task 2019-02-16T10:59:17.561Z] 10:59:17 INFO - REFTEST INFO | Unexpected: 0 (0 unexpected fail, 0 unexpected pass, 0 unexpected asserts, 0 failed load, 0 exception)
[task 2019-02-16T10:59:17.562Z] 10:59:17 INFO - REFTEST INFO | Known problems: 0 (0 known fail, 0 known asserts, 0 random, 0 skipped, 0 slow)
[task 2019-02-16T10:59:17.562Z] 10:59:17 INFO - REFTEST SUITE-END | Shutdown
[task 2019-02-16T10:59:17.563Z] 10:59:17 INFO - REFTEST INFO | :::
[task 2019-02-16T10:59:17.563Z] 10:59:17 INFO - REFTEST INFO | ::: Test verification summary for:
[task 2019-02-16T10:59:17.576Z] 10:59:17 INFO - REFTEST INFO | :::
[task 2019-02-16T10:59:17.577Z] 10:59:17 INFO - REFTEST INFO | ::: /builds/worker/workspace/build/tests/reftest/tests/layout/reftests/css-mediaqueries/greenbox-print.html
[task 2019-02-16T10:59:17.578Z] 10:59:17 INFO - REFTEST INFO | :::
[task 2019-02-16T10:59:17.578Z] 10:59:17 INFO - REFTEST INFO | ::: 1. Run each test 10 times in one browser. : Pass
[task 2019-02-16T10:59:17.579Z] 10:59:17 INFO - REFTEST INFO | ::: 2. Run each test 5 times in a new browser each time. : Pass
[task 2019-02-16T10:59:17.580Z] 10:59:17 INFO - REFTEST INFO | ::: 3. Run each test 10 times in one browser, in chaos mode. : Pass
[task 2019-02-16T10:59:17.580Z] 10:59:17 INFO - REFTEST INFO | ::: 4. Run each test 5 times in a new browser each time, in chaos mode. : Pass
[task 2019-02-16T10:59:17.581Z] 10:59:17 INFO - REFTEST INFO | :::
[task 2019-02-16T10:59:17.581Z] 10:59:17 INFO - REFTEST INFO | ::: Test verification PASSED
[task 2019-02-16T10:59:17.582Z] 10:59:17 INFO - REFTEST INFO | :::
[task 2019-02-16T10:59:17.627Z] 10:59:17 INFO - Return code: 0
[task 2019-02-16T10:59:17.629Z] 10:59:17 ERROR - No checks run.
[task 2019-02-16T10:59:17.630Z] 10:59:17 INFO - TinderboxPrint: reftest-reftest<br/><em class="testfail">T-FAIL</em>
[task 2019-02-16T10:59:17.631Z] 10:59:17 ERROR - # TBPL FAILURE #
[task 2019-02-16T10:59:17.632Z] 10:59:17 WARNING - setting return code to 2
[task 2019-02-16T10:59:17.633Z] 10:59:17 ERROR - TinderboxPrint: Per-test run of .../greenbox-print.html<br/>: FAILURE
[task 2019-02-16T10:59:17.634Z] 10:59:17 INFO - Running command: ['/builds/worker/workspace/build/venv/bin/python', '-u', '/builds/worker/workspace/build/tests/reftest/runreftest.py', '--appname=/builds/worker/workspace/build/application/firefox/firefox', '--utility-path=tests/bin', '--extra-profile-file=tests/bin/plugins', '--symbols-path=https://queue.taskcluster.net/v1/task/CHlqsxhGTCuVOdQPBn0stQ/artifacts/public/build/target.crashreporter-symbols.zip', '--log-raw=/builds/worker/workspace/build/blobber_upload_dir/reftest_raw.log', '--log-errorsummary=/builds/worker/workspace/build/blobber_upload_dir/reftest_errorsummary.log', '--cleanup-crashes', '--marionette-startup-timeout=180', '--sandbox-read-whitelist=/builds/worker/workspace/build', '--log-raw=-', '--suite=reftest', '--setpref=layers.acceleration.force-enabled=true', '--verify-max-time=900', '--verify', u'/builds/worker/workspace/build/tests/reftest/tests/layout/reftests/css-mediaqueries/mq_prefers_color_scheme_dark.html'] in /builds/worker/workspace/build
[task 2019-02-16T10:59:17.635Z] 10:59:17 INFO - Copy/paste: /builds/worker/workspace/build/venv/bin/python -u /builds/worker/workspace/build/tests/reftest/runreftest.py --appname=/builds/worker/workspace/build/application/firefox/firefox --utility-path=tests/bin --extra-profile-file=tests/bin/plugins --symbols-path=https://queue.taskcluster.net/v1/task/CHlqsxhGTCuVOdQPBn0stQ/artifacts/public/build/target.crashreporter-symbols.zip --log-raw=/builds/worker/workspace/build/blobber_upload_dir/reftest_raw.log --log-errorsummary=/builds/worker/workspace/build/blobber_upload_dir/reftest_errorsummary.log --cleanup-crashes --marionette-startup-timeout=180 --sandbox-read-whitelist=/builds/worker/workspace/build --log-raw=- --suite=reftest --setpref=layers.acceleration.force-enabled=true --verify-max-time=900 --verify /builds/worker/workspace/build/tests/reftest/tests/layout/reftests/css-mediaqueries/mq_prefers_color_scheme_dark.html
[task 2019-02-16T10:59:17.636Z] 10:59:17 INFO - Using env: (same as previous command)
[task 2019-02-16T10:59:17.637Z] 10:59:17 INFO - Calling ['/builds/worker/workspace/build/venv/bin/python', '-u', '/builds/worker/workspace/build/tests/reftest/runreftest.py', '--appname=/builds/worker/workspace/build/application/firefox/firefox', '--utility-path=tests/bin', '--extra-profile-file=tests/bin/plugins', '--symbols-path=https://queue.taskcluster.net/v1/task/CHlqsxhGTCuVOdQPBn0stQ/artifacts/public/build/target.crashreporter-symbols.zip', '--log-raw=/builds/worker/workspace/build/blobber_upload_dir/reftest_raw.log', '--log-errorsummary=/builds/worker/workspace/build/blobber_upload_dir/reftest_errorsummary.log', '--cleanup-crashes', '--marionette-startup-timeout=180', '--sandbox-read-whitelist=/builds/worker/workspace/build', '--log-raw=-', '--suite=reftest', '--setpref=layers.acceleration.force-enabled=true', '--verify-max-time=900', '--verify', u'/builds/worker/workspace/build/tests/reftest/tests/layout/reftests/css-mediaqueries/mq_prefers_color_scheme_dark.html'] with output_timeout 1000
[task 2019-02-16T10:59:17.916Z] 10:59:17 INFO - REFTEST INFO | :::
[task 2019-02-16T10:59:17.917Z] 10:59:17 INFO - REFTEST INFO | ::: Running test verification step "1. Run each test 10 times in one browser."...
[task 2019-02-16T10:59:17.918Z] 10:59:17 INFO - REFTEST INFO | :::
[task 2019-02-16T10:59:17.919Z] 10:59:17 INFO - REFTEST INFO | Removed pending crash reports at '/builds/worker/.mozilla/firefox/Crash Reports'
[task 2019-02-16T10:59:17.960Z] 10:59:17 INFO - REFTEST INFO | Running with e10s: True
[task 2019-02-16T10:59:17.961Z] 10:59:17 INFO - REFTEST INFO | Application command: /builds/worker/workspace/build/application/firefox/firefox -marionette -profile /tmp/tmpd8p8yN.mozrunner
[task 2019-02-16T10:59:18.516Z] 10:59:18 INFO - 1550314758510 addons.webextension.screenshots@mozilla.org WARN Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid extension permission: mozillaAddons
[task 2019-02-16T10:59:18.517Z] 10:59:18 INFO - 1550314758511 addons.webextension.screenshots@mozilla.org WARN Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid extension permission: resource://pdf.js/
[task 2019-02-16T10:59:18.518Z] 10:59:18 INFO - 1550314758511 addons.webextension.screenshots@mozilla.org WARN Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid extension permission: about:reader*

Something in this merge is causing the failure https://treeherder.mozilla.org/#/jobs?repo=autoland&selectedJob=228809744&revision=939d6ac72fedd803d22d3f62a995cb46b7c2b825

Retriggers on both autoland and inbound before this merge are green.

Bug 1494034 modified layout/reftests/css-mediaqueries/greenbox-print.html, which triggered TVg. On initial landing on inbound, all TV tasks passed:

https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&revision=4739353088fcad81bc126deab56e8de0fad7a88a&searchStr=verify&selectedJob=228734700

Closer examination shows this particular test resulted in no testing:

13:28:05 INFO - REFTEST SUITE-START | Running 0 tests
13:28:05 INFO - REFTEST INFO | Result summary:
13:28:05 INFO - REFTEST INFO | Successful: 0 (0 pass, 0 load only)
13:28:05 INFO - REFTEST INFO | Unexpected: 0 (0 unexpected fail, 0 unexpected pass, 0 unexpected asserts, 0 failed load, 0 exception)
13:28:05 INFO - REFTEST INFO | Known problems: 0 (0 known fail, 0 known asserts, 0 random, 0 skipped, 0 slow)
13:28:05 INFO - REFTEST SUITE-END | Shutdown
13:28:05 INFO - REFTEST INFO | :::
13:28:05 INFO - REFTEST INFO | ::: Test verification summary for:
13:28:05 INFO - REFTEST INFO | :::
13:28:05 INFO - REFTEST INFO | ::: /Users/cltbld/tasks/task_1550265534/build/tests/reftest/tests/layout/reftests/css-mediaqueries/greenbox-print.html
13:28:05 INFO - REFTEST INFO | :::
13:28:05 INFO - REFTEST INFO | ::: 1. Run each test 10 times in one browser. : Pass
13:28:05 INFO - REFTEST INFO | ::: 2. Run each test 5 times in a new browser each time. : Pass
13:28:05 INFO - REFTEST INFO | ::: 3. Run each test 10 times in one browser, in chaos mode. : Pass
13:28:05 INFO - REFTEST INFO | ::: 4. Run each test 5 times in a new browser each time, in chaos mode. : Pass
13:28:05 INFO - REFTEST INFO | :::
13:28:05 INFO - REFTEST INFO | ::: Test verification PASSED
13:28:05 INFO - REFTEST INFO | :::
13:28:05 INFO - Return code: 0

At issue is that greenbox-print.html is a reference file, so when the reftest harness is asked to run tests against it, no tests are run.

TV normally guards against this by trying to guess which files are reference files, but that's not working in this case:

https://searchfox.org/mozilla-central/rev/4587d146681b16ff9878a6fdcba53b04f76abe1d/testing/mozharness/mozharness/mozilla/testing/per_test_base.py#305-322

Tier 2 and non-recurring -- clearing needswork.

Why did this fail on the merge, but not on original landing?

On original landing (inbound):

[task 2019-02-15T21:28:31.272Z] 21:28:31 INFO - REFTEST INFO | ::: Test verification PASSED
[task 2019-02-15T21:28:31.273Z] 21:28:31 INFO - REFTEST INFO | :::
[task 2019-02-15T21:28:31.303Z] 21:28:31 INFO - Return code: 0
[task 2019-02-15T21:28:31.305Z] 21:28:31 INFO - TinderboxPrint: reftest-reftest<br/><em class="testfail">T-FAIL</em>
[task 2019-02-15T21:28:31.306Z] 21:28:31 WARNING - # TBPL SUCCESS #
[task 2019-02-15T21:28:31.306Z] 21:28:31 WARNING - TinderboxPrint: Per-test run of .../greenbox-print.html<br/>: SUCCESS

On merge (autoland):

[task 2019-02-16T10:59:17.581Z] 10:59:17 INFO - REFTEST INFO | ::: Test verification PASSED
[task 2019-02-16T10:59:17.582Z] 10:59:17 INFO - REFTEST INFO | :::
[task 2019-02-16T10:59:17.627Z] 10:59:17 INFO - Return code: 0
[task 2019-02-16T10:59:17.629Z] 10:59:17 ERROR - No checks run.
[task 2019-02-16T10:59:17.630Z] 10:59:17 INFO - TinderboxPrint: reftest-reftest<br/><em class="testfail">T-FAIL</em>
[task 2019-02-16T10:59:17.631Z] 10:59:17 ERROR - # TBPL FAILURE #
[task 2019-02-16T10:59:17.632Z] 10:59:17 WARNING - setting return code to 2
[task 2019-02-16T10:59:17.633Z] 10:59:17 ERROR - TinderboxPrint: Per-test run of .../greenbox-print.html<br/>: FAILURE

I think "No checks run" comes from

https://searchfox.org/mozilla-central/rev/4587d146681b16ff9878a6fdcba53b04f76abe1d/testing/mozharness/mozharness/mozilla/structuredlog.py#204-211

Possibly the inbound case succeeds because greenbox-print.html was the second test to be verified, and by that point in the log there was already a test_end for the previous test; on merges, greenbox-print.html was the first test to be verified.

(In reply to Intermittent Failures Robot from comment #8)

In this case, only osx/opt failed

https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=3d19ee7bc2d7967b730aab39594f81eb3e0c3626&selectedJob=232552547&searchStr=verify

The failed test is subject to an unusual skip-if:

https://searchfox.org/mozilla-central/rev/89414a1df52d06cfc35529afb9a5a8542a6e4270/devtools/client/webreplay/mochitest/browser.ini#5

skip-if = os != "mac" || debug || !nightly_build

The per-test mozinfo thinks this is a nightly_build, in agreement with the build's mozinfo:

23:58:18     INFO - Per-test run using mozinfo: {u'bin_suffix': u'', u'official': True, u'allow_legacy_extensions': True, u'pgo': False, u'sync': True, u'buildapp': u'browser', u'crashreporter': True, u'devedition': False, u'require_signing': False, u'platform_guess': u'macosx64', u'appname': u'firefox', u'stylo': True, u'mozconfig': u'/builds/worker/workspace/build/src/.mozconfig', u'topsrcdir': u'/builds/worker/workspace/build/src', u'ubsan': False, 'os_version': StringVersion ('10.10'), 'version': 'OS X 10.10.5', 'webrender': False, u'buildtype_guess': u'opt', 'bits': 64, 'has_sandbox': True, u'datareporting': True, u'artifact': False, u'toolkit': u'cocoa', u'healthreport': True, u'updater': True, u'cc_type': u'clang', u'asan': False, u'release_or_beta': False, u'ccov': False, u'tests_enabled': True, 'e10s': True, u'telemetry': False, u'tsan': False, u'nightly_build': True, 'headless': False, 'verify': True, u'debug': False, 'os': u'mac', 'processor': u'x86_64'}

so it makes sense that TV tried to run the test...but why wouldn't the harness?

00:10:45     INFO - Calling ['/Users/cltbld/tasks/task_1552003035/build/venv/bin/python', '-u', '/Users/cltbld/tasks/task_1552003035/build/tests/mochitest/runtests.py', '--appname=/Users/cltbld/tasks/task_1552003035/build/application/Firefox Nightly.app/Contents/MacOS/firefox', '--utility-path=tests/bin', '--extra-profile-file=tests/bin/plugins', '--symbols-path=https://queue.taskcluster.net/v1/task/FerjBvMcTsW1-Pi_vXV0_Q/artifacts/public/build/target.crashreporter-symbols.zip', '--certificate-path=tests/certs', '--quiet', '--log-raw=/Users/cltbld/tasks/task_1552003035/build/blobber_upload_dir/browser-chrome_raw.log', '--log-errorsummary=/Users/cltbld/tasks/task_1552003035/build/blobber_upload_dir/browser-chrome_errorsummary.log', '--screenshot-on-fail', '--cleanup-crashes', '--marionette-startup-timeout=180', '--sandbox-read-whitelist=/Users/cltbld/tasks/task_1552003035/build', '--log-raw=-', '--flavor=browser', '--verify-max-time=900', '--verify', u'devtools/client/webreplay/mochitest/browser_dbg_rr_logpoint-02.js'] with output_timeout 1000
00:10:45     INFO - :::
00:10:45     INFO - ::: Running test verification step "1. Run each test 10 times in one browser."...
00:10:45     INFO - :::
00:10:45     INFO - Checking for ssltunnel processes...
00:10:45     INFO - Checking for xpcshell processes...
00:10:45     INFO - mozcrash Removed pending crash reports at '/Users/cltbld/Library/Application Support/firefox/Crash Reports'
00:10:46    ERROR - No tests were found for flavor 'browser' and the following manifest filters:
00:10:46    ERROR - skip_if, run_if, fail_if, remove_imptest_failure_expectations, subsuite(name=None), pathprefix(['devtools/client/webreplay/mochitest/browser_dbg_rr_logpoint-02.js'])

The recent spike was triggered by bug 1440537.

https://taskcluster-artifacts.net/dmuH_kXXSImebDPwMF6Qog/0/public/logs/live_backing.log

12:07:18 INFO - Per-test run found test accessible/tests/mochitest/aom/test_general.html (mochitest-a11y/None)
12:07:18 INFO - Per-test run found test devtools/client/shared/test/browser_html_tooltip-02.js (mochitest-browser-chrome/devtools)
12:07:18 INFO - 'layout/style/test/test_media_queries.html' has been skipped on this platform.
12:07:18 INFO - Per-test run found test toolkit/components/extensions/test/mochitest/test_ext_new_tab_processType.html (mochitest-plain/None)

12:09:00 INFO - Calling ['/Users/cltbld/tasks/task_1556365878/build/venv/bin/python', '-u', '/Users/cltbld/tasks/task_1556365878/build/tests/mochitest/runtests.py', '--appname=/Users/cltbld/tasks/task_1556365878/build/application/Firefox Nightly.app/Contents/MacOS/firefox', '--utility-path=tests/bin', '--extra-profile-file=tests/bin/plugins', u'--symbols-path=https://queue.taskcluster.net/v1/task/FKGYUtxqS0uvN4yPaMBjjQ/artifacts/public/build/target.crashreporter-symbols.zip', '--certificate-path=tests/certs', '--quiet', '--log-raw=/Users/cltbld/tasks/task_1556365878/build/blobber_upload_dir/mochitest-a11y_raw.log', '--log-errorsummary=/Users/cltbld/tasks/task_1556365878/build/blobber_upload_dir/mochitest-a11y_errorsummary.log', '--screenshot-on-fail', '--cleanup-crashes', '--marionette-startup-timeout=180', '--sandbox-read-whitelist=/Users/cltbld/tasks/task_1556365878/build', '--log-raw=-', '--flavor=a11y', '--verify-max-time=900', '--verify', u'accessible/tests/mochitest/aom/test_general.html'] with output_timeout 1000
12:09:00 INFO - usage: runtests.py [options] [test paths]
12:09:00 INFO - runtests.py: error: mochitest-a11y does not support e10s, try again with --disable-e10s.
12:09:01 ERROR - Return code: 2
12:09:01 ERROR - No checks run.
12:09:01 ERROR - No suite end message was emitted by this harness.
12:09:01 INFO - TinderboxPrint: mochitest-mochitest-a11y<br/><em class="testfail">T-FAIL</em>
12:09:01 ERROR - # TBPL FAILURE #
12:09:01 WARNING - setting return code to 2

Hopefully this is caused by bug 1545414.

Reproduced on try: https://treeherder.mozilla.org/#/jobs?repo=try&tier=1%2C2%2C3&revision=9effddff82aea671a4d0b0c06fd3601d0c58abf7

12:09:00 INFO - runtests.py: error: mochitest-a11y does not support e10s, try again with --disable-e10s.

This is from bug 1546100: now when flavor is a11y or chrome, --disable-e10s is required.

Recent failures:

55950b79d6b9204d8340e1b7e5458131869a0c3e, 38a326f813f6b1fd1400d215730e0105f62eb9c5, ad04ccedc21ed4373acff2d310388bc55725182b - mochitest-chrome - bug 1548392

c89cf3ac460f35aa4c0516410c25e88e0f6cc8e6 - mis-classification

75a329d3bce4f654214b1d14c2cfb6b22770e8ee, 2a26f848ec318a586ebe478651b0e20d72afc337, ee37c856a47e46f3f548cef19d64b581119188fb - mochitest-chrome - bug 1548392

5257cabf18a9f322ff48973f2e02cc830eba5aa1, 70a7b7db7b84452a83026949fccda6d10ca57869, b1ca002764784802c57f7b13e4d4971a4b67ba72 - mochitest-a11y - bug 1548392

Geoff there are 95 total failures since May 2nd when you posted the above comment.

Can we assume this will be fixed with bug 1548392? I saw you posted a patch there.

(In reply to Andreea Pavel [:apavel] from comment #22)

Can we assume this will be fixed with bug 1548392? I saw you posted a patch there.

It might not be 100% fixed, but all of the recent failures would be avoided, so I'm confident this bug would be at least 90% fixed once the patch on bug 1548392 lands.

Thank you Geoff.

After bug 1548392 there were no new failures for a few days.

But today there are, all on Windows, all on TVg. Many files are involved, all in layout/reftests/w3c-css/received, such as:

layout/reftests/w3c-css/received/css-writing-modes/table-progression-vlr-002.html
layout/reftests/w3c-css/received/css-multicol/multicol-nested-002.xht
layout/reftests/w3c-css/received/css-multicol/multicol-columns-007.xht
layout/reftests/w3c-css/received/css-multicol/multicol-nested-margin-003.xht
...

TV was triggered by changes in bug 1547162.

These files are present in the manifest layout/reftests/w3c-css/received/reftest.list and appear to be valid reftests, but the harness is not running them -- on windows only.

Waiting for bug: 1550808

The most recent failures were caused by trying to verify a reference reftest file, on all platforms; I wonder if the test-vs-reference file issue can be properly resolved now.

(In reply to Intermittent Failures Robot from comment #35)

6 failures in 3964 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

These were:

https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=271192383&repo=autoland&lineNumber=1162
[task 2019-10-14T19:29:06.057Z] 19:29:06 INFO - Per-test run found test dom\imptests\failures\html\typedarrays\test_constructors.html.json (mochitest-plain/None)

https://hg.mozilla.org/integration/autoland/rev/b4c5c8d78d5f9bfbbaacc34929ff6bece064cc44 added that file as a test file in mochitest.ini; I assume that was an error in the patch. At any rate, the errant test manifest (and test) have been removed from the tree now.

I don't think TV can do much to handle this situation better, except perhaps to generate a more explanatory error message.

(In reply to Intermittent Failures Robot from comment #38)

18 failures in 5444 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

All of these failures were associated with the same revision, which included a backout of bug 1594937; the backout restored (added) many reftests.

The first reftest attempted was /builds/worker/workspace/build/tests/reftest/tests/layout/reftests/w3c-css/received/css-multicol/css-multicol/multicol-clip-001.xht, and it failed:

[task 2019-11-23T07:10:16.139Z] 07:10:16     INFO - Calling ['/builds/worker/workspace/build/venv/bin/python', '-u', '/builds/worker/workspace/build/tests/reftest/runreftest.py', '--setpref=media.peerconnection.mtransport_process=false', '--setpref=network.process.enabled=false', '--appname=/builds/worker/workspace/build/application/firefox/firefox', '--utility-path=tests/bin', '--extra-profile-file=tests/bin/plugins', u'--symbols-path=https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/OVwRw7MsTKK2A7-bWaU_Uw/artifacts/public/build/target.crashreporter-symbols.zip', '--log-raw=/builds/worker/workspace/build/blobber_upload_dir/reftest_raw.log', '--log-errorsummary=/builds/worker/workspace/build/blobber_upload_dir/reftest_errorsummary.log', '--cleanup-crashes', '--marionette-startup-timeout=180', '--sandbox-read-whitelist=/builds/worker/workspace/build', '--log-raw=-', '--suite=reftest', '--setpref=layers.acceleration.force-enabled=true', '--verify-max-time=900', '--verify', u'/builds/worker/workspace/build/tests/reftest/tests/layout/reftests/w3c-css/received/css-multicol/css-multicol/multicol-clip-001.xht'] with output_timeout 1000
...
[task 2019-11-23T07:10:22.283Z] 07:10:22     INFO - REFTEST SUITE-START | Running 0 tests

The path '.../css-multicol/css-multicol/...' seems unusual, and I do not see that path in https://hg.mozilla.org/integration/autoland/rev/4a51a464d8efabde6a41af89fbb6aace77613f98 -- very curious...

(This push also backed out bug 1583364, which may have had an effect on the test resolver...but I don't see anything wrong there.)

[task 2019-11-23T05:59:56.742Z] 05:59:56     INFO - Per-test run substituting layout/reftests/w3c-css/received/css-multicol/css-multicol/multicol-collapsing-001.xht for layout/reftests/w3c-css/received/css-multicol/multicol-collapsing-001-ref.xht

https://searchfox.org/mozilla-central/rev/0678172d5b5c681061b904c776b668489e3355b0/testing/mozharness/mozharness/mozilla/testing/per_test_base.py#139