Filed by: ccoroiu [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=253699986&repo=mozilla-central
Full log: https://queue.taskcluster.net/v1/task/ZHf6zlvQRo-VcXLstv5XBQ/runs/0/artifacts/public/logs/live_backing.log
Reftest URL: https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://queue.taskcluster.net/v1/task/ZHf6zlvQRo-VcXLstv5XBQ/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1

11:56:12 INFO - REFTEST SUITE-START | Running 0 tests
11:56:12 INFO - REFTEST INFO | Result summary:
11:56:12 INFO - REFTEST INFO | Successful: 0 (0 pass, 0 load only)
11:56:12 INFO - REFTEST INFO | Unexpected: 0 (0 unexpected fail, 0 unexpected pass, 0 unexpected asserts, 0 failed load, 0 exception)
11:56:12 INFO - REFTEST INFO | Known problems: 0 (0 known fail, 0 known asserts, 0 random, 0 skipped, 0 slow)
11:56:12 INFO - REFTEST SUITE-END | Shutdown
11:56:12 INFO - REFTEST INFO | :::
11:56:12 INFO - REFTEST INFO | ::: Test verification summary for:
11:56:12 INFO - REFTEST INFO | :::
11:56:12 INFO - REFTEST INFO | ::: Z:/task_1561632535/build/tests/reftest/tests/layout/reftests/details-summary/overflow-hidden-details.html
11:56:12 INFO - REFTEST INFO | :::
11:56:12 INFO - REFTEST INFO | ::: 1. Run each test 10 times in one browser. : Pass
11:56:12 INFO - REFTEST INFO | ::: 2. Run each test 5 times in a new browser each time. : Pass
11:56:12 INFO - REFTEST INFO | ::: 3. Run each test 10 times in one browser, in chaos mode. : Pass
11:56:12 INFO - REFTEST INFO | ::: 4. Run each test 5 times in a new browser each time, in chaos mode. : Pass
11:56:12 INFO - REFTEST INFO | :::
11:56:12 INFO - REFTEST INFO | ::: Test verification PASSED
11:56:12 INFO - REFTEST INFO | :::
11:56:12 INFO - Return code: 0
11:56:12 INFO - TinderboxPrint: reftest-reftest<br/><em class="testfail">T-FAIL</em>
11:56:12 ERROR - # TBPL FAILURE #
11:56:12 WARNING - setting return code to 2
11:56:12 ERROR - TinderboxPrint: Per-test run of ...\overflow-hidden-details.html<br/>: FAILURE
11:56:12 INFO - Running command: ['Z:\task_1561632535\build\venv\Scripts\python', '-u', 'Z:\task_1561632535\build\tests\reftest\runreftest.py', '--appname=Z:\task_1561632535\build\application\firefox\firefox.exe', '--utility-path=tests/bin', '--extra-profile-file=tests/bin/plugins', u'--symbols-path=https://queue.taskcluster.net/v1/task/c11IObXKT56ONzINw4MFsQ/artifacts/public/build/target.crashreporter-symbols.zip', '--log-raw=Z:\task_1561632535\build\blobber_upload_dir\reftest_raw.log', '--log-errorsummary=Z:\task_1561632535\build\blobber_upload_dir\reftest_errorsummary.log', '--cleanup-crashes', '--marionette-startup-timeout=180', '--sandbox-read-whitelist=Z:\task_1561632535\build', '--log-raw=-', '--suite=reftest', '--verify-max-time=900', '--verify', u'Z:/task_1561632535/build/tests/reftest/tests/layout/reftests/details-summary/overflow-auto-open-details.html'] in Z:\task_1561632535\build
11:56:12 INFO - Copy/paste: Z:\task_1561632535\build\venv\Scripts\python -u Z:\task_1561632535\build\tests\reftest\runreftest.py --appname=Z:\task_1561632535\build\application\firefox\firefox.exe --utility-path=tests/bin --extra-profile-file=tests/bin/plugins --symbols-path=https://queue.taskcluster.net/v1/task/c11IObXKT56ONzINw4MFsQ/artifacts/public/build/target.crashreporter-symbols.zip --log-raw=Z:\task_1561632535\build\blobber_upload_dir\reftest_raw.log --log-errorsummary=Z:\task_1561632535\build\blobber_upload_dir\reftest_errorsummary.log --cleanup-crashes --marionette-startup-timeout=180 --sandbox-read-whitelist=Z:\task_1561632535\build --log-raw=- --suite=reftest --verify-max-time=900 --verify Z:/task_1561632535/build/tests/reftest/tests/layout/reftests/details-summary/overflow-auto-open-details.html
11:56:12 INFO - Using env: (same as previous command)
11:56:12 INFO - Calling ['Z:\task_1561632535\build\venv\Scripts\python', '-u', 'Z:\task_1561632535\build\tests\reftest\runreftest.py', '--appname=Z:\task_1561632535\build\application\firefox\firefox.exe', '--utility-path=tests/bin', '--extra-profile-file=tests/bin/plugins', u'--symbols-path=https://queue.taskcluster.net/v1/task/c11IObXKT56ONzINw4MFsQ/artifacts/public/build/target.crashreporter-symbols.zip', '--log-raw=Z:\task_1561632535\build\blobber_upload_dir\reftest_raw.log', '--log-errorsummary=Z:\task_1561632535\build\blobber_upload_dir\reftest_errorsummary.log', '--cleanup-crashes', '--marionette-startup-timeout=180', '--sandbox-read-whitelist=Z:\task_1561632535\build', '--log-raw=-', '--suite=reftest', '--verify-max-time=900', '--verify', u'Z:/task_1561632535/build/tests/reftest/tests/layout/reftests/details-summary/overflow-auto-open-details.html'] with output_timeout 1000
11:56:12 INFO - REFTEST INFO | :::
11:56:12 INFO - REFTEST INFO | ::: Running test verification step "1. Run each test 10 times in one browser."...

This is a regression of Bug 1552578, Emilio can you please take a look?

So my patch touched that file, and it seems like test-verify tried to run it. But this is not a test but a reference file, so of course running it doesn't make sense, and thus the error (I guess).

Joel / Geoff, do you know what's going on here?

Yes, TV has trouble distinguishing reftest test vs reference files. We previously tried to address this in bug 1442790 and bug 1423771, and I thought those efforts were effective. Perhaps something in the reftest harness changed?

Anyway, not a concern for your push, :emilio -- sorry for the trouble!

No other occurrences since the 28th of June.

https://treeherder.mozilla.org/#/jobs?repo=try&revision=61682f23fd60409cd607412703ac43667d49856d verifies that this case is addressed by bug 1592754.