thanks to the work in bug 1465117, it is easier to add a single test name via a custom action in treeherder and retrigger a job on X revisions to run that test. The intention was to run this in test-verify and I had not thought it through and asked for the wrong thing. This bug is to do what I originally had intended. What needs to happen: 1) on retriggering the task when we enter a path in testPaths, we will append it to the env via MOZHARNESS_TEST_PATHS (as it currently does) 2) edit the command to run a generic test in --verify mode 3) adjust the label to be test-verify 4) adjust some minor args for index:rank, and chunks to keep taskcluster organized well 5) adjust the values for treeherder to have the right group/label/code. there are a lot of changes, I imagine I will get some things wrong here in the first pass.

an example try run (in progress): https://treeherder.mozilla.org/#/jobs?repo=try&revision=e189f4997d142c53b7cffd4179e359ea9d6207ed more interesting there is the code: https://hg.mozilla.org/try/rev/6ce4c36ca717c42f0d894e38c447fac55b144440 ideally a few tweaks and tests and we can get this landed.

there seems to be a lot to consider when hacking on a task. Things like different taskcluster workers treating commands differently, and taskcluster inserting build taskID dynamically into certain parts of the command to execute (installer-url for example). I did get this working: https://treeherder.mozilla.org/#/jobs?repo=try&revision=4c6f3eef1cab55d21609819972ffe2232d27b320 It is hacky, and I suspect many use cases will not work. When I have tested it on a larger set of tests (plain, gpu, wpt) and configs (linux, osx, windows, android), I will solicit feedback. meanwhile, if there are any drive by comments on my changes so far, they are welcome: https://hg.mozilla.org/try/rev/32493a0a36a97ad371d837fa3667d31e96bc365b I will sneak in a small change to this to fix the --geckoProfile as well- the original patch added the string as an env variable, and it needs to be added to the command line. I will consider using the env var instead of hacking the cli.

this is ugly and sloppy, not sure of a cleaner way to do this, but open to suggestions.

on my recent push, I tested a good bit on try: https://treeherder.mozilla.org/#/jobs?repo=try&revision=f02781710d5a049d07d9557405490b28080fd098&filter-tier=1&filter-tier=2&filter-tier=3 for the different tests on all 5 operating systems, I tested: reftest: layout/reftests/writing-mode/tables/vertical-table-1a.html xpcshell: chrome/test/unit/test_bug292789.js wpt: testing/web-platform/tests/WebCryptoAPI/generateKey/failures_AES-CBC.https.any.js webgl: dom/canvas/test/crash/test_1251091-1.html mochitest: dom/indexedDB/test/test_count.html jsreftest: js/src/tests/non262/Array/regress-101964.js * no test run on android, not sure why mochitest-media: dom/media/test/test_VideoPlaybackQuality_disabled.html * not run on android-hw (this is new bitbar, happy to investigate later) you can see a couple scenarios here are not met, I believe we can tackle those in follow up issues. I did shift slightly to a model where we define gpu_required by inspecting the test paths instead of a hybrid of task name and test path. Also I need to test all of this in >1 test provided, although I see that as a different use case altogether. Open to further thoughts.

Comment on attachment 8989080 [details] Bug 1471227 - adjust taskcluster backfill with path to use test-verify instead of original job. r=gbrown Geoff Brown [:gbrown] has approved the revision. https://phabricator.services.mozilla.com/D1908

Comment on attachment 8989841 [details] Bug 1471227 - adjust taskcluster backfill with path to use test-verify instead of original job. r=gbrown carry over from previous patch

Comment on attachment 8989841 [details] Bug 1471227 - adjust taskcluster backfill with path to use test-verify instead of original job. r=gbrown Geoff Brown [:gbrown] has approved the revision. Joel Maher ( :jmaher ) (UTC-4) has been removed from the revision. https://phabricator.services.mozilla.com/D1961

Pushed by jmaher@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/22764fd6ae90 adjust taskcluster backfill with path to use test-verify instead of original job. r=gbrown