There's a Treeherder action feature (pulse_actions + mozci) which is used by sheriffs for merging commits around. Trigger missing jobs analyzes which jobs have not been scheduled on a push and schedules them. I'm happy to make this my deliverable for this quarter. I will need just a bit of direction as to what code to touch. Any thoughts on how to implement this? Is an action task still a good way for implementing it?

Please implement in actions.json. I'm working on fleshing out support for that in Treeherder now (it's there but sort of limited at the moment).

I believe there's a typo on the documentation saying that tasks should be defined as taskcluster/taskgraph/actions/my-action.py instead of taskcluster/actions/my-action.py. What command do I use to generate an actions.json file locally? The one a decision task executes [1]? Is there also a command to test a created task directly from my local machine? I could not find it on the documentation. Also, do you have any suggestion on how or where to trasverse the task graph to determine what was scheduled and what was not? Is this information something I can discover through some of the artifacts of the gecko decision task? Thanks in advance. [1] ./mach --log-no-times taskgraph decision --pushlog-id=32145 --pushdate=1499443022 --project=mozilla-central '--message= ' --owner=ffxbld@noreply.mozilla.org --level=3 --base-repository=https://hg.mozilla.org/mozilla-central --head-repository=https://hg.mozilla.org/mozilla-central/ --head-ref=5eeee16f1659ca1f7c4e28e5834d051eb060eafc --head-rev=5eeee16f1659ca1f7c4e28e5834d051eb060eafc

bstack will be looking at older style actions tasks this week and describe what we will need for this. The following is some of what we discussed: armenzg said: 1) create a mach command that can determine what is missing and schedule them 2) add an action py file under taskcluster/actions that would call that mach command 3) modify TH to use actions.json and show push/resulset level actions bstack said: "I'm not sure we need it to be a mach thing, but it probably isn't an issue either way"

Treeherder support of actions.json for job-level actions was added here: https://github.com/mozilla/treeherder/commit/a45f2d3c48c0b6e79fe26609d1599558ad91c44d

I'll see if I can work on this.

Note that this assumes that any test dependencies were also built. We don't currently optimize away builds when tests are optimized away, so I don't think that will be a problem. If it is, it's just a little more logic to add to expand `to_run` with any tasks that are dependencies of other tasks in to_run and that have not themselves been run.

[task 2017-07-21T13:18:51.519609Z] ImportError: No module named util [task 2017-07-21T13:18:51.519628Z] [task 2017-07-21T13:18:51.519679Z] File "/home/worker/checkouts/gecko/taskcluster/mach_commands.py", line 150, in taskgraph_decision [task 2017-07-21T13:18:51.519716Z] import taskgraph.decision

I forgot, mach gives exceptions backward. Reversing, and filtering out the custom import it makes a lot more sense: [task 2017-07-21T13:18:51.519609Z] ImportError: No module named util [task 2017-07-21T13:18:51.519628Z] [task 2017-07-21T13:18:51.520442Z] File "/home/worker/checkouts/gecko/taskcluster/actions/run_missing_tests.py", line 13, in <module> [task 2017-07-21T13:18:51.520477Z] from actions.util import create_task [task 2017-07-21T13:18:51.520258Z] File "/home/worker/checkouts/gecko/taskcluster/actions/registry.py", line 317, in <module> [task 2017-07-21T13:18:51.520291Z] __import__('actions.' + f[:-3]) [task 2017-07-21T13:18:51.520073Z] File "/home/worker/checkouts/gecko/taskcluster/actions/__init__.py", line 9, in <module> [task 2017-07-21T13:18:51.520112Z] from .registry import ( [task 2017-07-21T13:18:51.519869Z] File "/home/worker/checkouts/gecko/taskcluster/taskgraph/decision.py", line 20, in <module> [task 2017-07-21T13:18:51.519908Z] from actions import render_actions_json [task 2017-07-21T13:18:51.519679Z] File "/home/worker/checkouts/gecko/taskcluster/mach_commands.py", line 150, in taskgraph_decision [task 2017-07-21T13:18:51.519716Z] import taskgraph.decision

Comment on attachment 8888490 [details] Bug 1379163: Implement the "Run Missing Tests" action https://reviewboard.mozilla.org/r/159452/#review165170 lgtm. Thanks Dustin!

sheriffs: FYI the trigger missing jobs will be slightly different than on Buildbot (at the implementation level) as TaskCluster always creates builds, thus, it will be looking at triggering test tasks that were optimized (skipped). This I believe should work for you. dustin: I was thinking; can you output the labels of the tasks that get scheduled? or will it be easy to figure out which tasks gets scheduled through other means? (e.g. looking at a task group or something similar). I want it easy to inspect a posteriori.

Yeah, I can add that, good thinking. Also, the exception above was because bug 1382707 hadn't landed yet. And note that its bugfix followup hasn't landed yet either, so at the moment actions don't work.

Ah, I see that it already does that (create_task has `logger.debug("Creating task with taskId {} for {}".format(task_id, label))`

https://treeherder.mozilla.org/#/jobs?repo=try&revision=cc6bf85e3e8f690c37ff5961f0a7cbf55792ffb3&selectedJob=116433890 https://public-artifacts.taskcluster.net/fTybOeR9RvusrNOmQHIxSQ/0/public/logs/live_backing.log is not very exciting, but you get the idea! From my local runs against m-c, this does in fact schedule tests when there are tests to schedule. To trigger this action: - select a task in treeherder - ... -> custom action - choose "run missing tests" - type "null" in the JSON payload form - click "Trigger"

Comment on attachment 8888815 [details] Bug 1379163: defer loading actions until needed; https://reviewboard.mozilla.org/r/159850/#review165280 Nice :)

Comment on attachment 8888488 [details] Bug 1379163: add missing license headers; https://reviewboard.mozilla.org/r/159448/#review165298

Comment on attachment 8888489 [details] Bug 1379163: make parameterization functions into utilities; https://reviewboard.mozilla.org/r/159450/#review165308

Comment on attachment 8888490 [details] Bug 1379163: Implement the "Run Missing Tests" action https://reviewboard.mozilla.org/r/159452/#review165380 Otherwise this makes a lot of sense! Let's talk Monday about which bits should be moved to util! ::: taskcluster/actions/run_missing_tests.py:46 (Diff revision 3) > +def run_missing_tests(parameters, input, task_group_id, task_id, task): > + decision_task_id = find_decision_task(parameters) > + > + full_task_graph = get_artifact(decision_task_id, "public/full-task-graph.json") > + _, full_task_graph = TaskGraph.from_json(full_task_graph) > + task_graph = get_artifact(decision_task_id, "public/task-graph.json") This is never used, right? I only see `full_task_graph` from here (and the line below it) on out.

Yeah, autoland will make our collaboration a little difficult. We could collaborate temporarily in a user repo.. Let me know when you want to talk.

Pushed by dmitchell@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/963b59bd61a3 add missing license headers; r=bstack https://hg.mozilla.org/integration/autoland/rev/2e143d567065 make parameterization functions into utilities; r=bstack https://hg.mozilla.org/integration/autoland/rev/aff22b63df7a Implement the "Run Missing Tests" action; r=armenzg,bstack https://hg.mozilla.org/integration/autoland/rev/03896a3c7a68 defer loading actions until needed; r=jonasfj

Yay!

https://hg.mozilla.org/mozilla-central/rev/963b59bd61a3 https://hg.mozilla.org/mozilla-central/rev/2e143d567065 https://hg.mozilla.org/mozilla-central/rev/aff22b63df7a https://hg.mozilla.org/mozilla-central/rev/03896a3c7a68

Follow ups are bug 1383066 and now bug 1388407.