One of the most confusing parts of task graphs is the fact that tasks can be included in a graph for unclear reasons. One of the common scenarios for this is a test whose run-on-projects has entries that the build it depends on does not. For example:
webgpu sets run-on-projects to ['trunk']: https://hg.mozilla.org/mozilla-central/file/71a05993c9bd5ff8c2e6e264096b73098033147c/taskcluster/ci/test/mochitest.yml#l435

But some of the builds it runs on, like linux64/opt, set it to ['mozilla-central']: https://hg.mozilla.org/mozilla-central/file/71a05993c9bd5ff8c2e6e264096b73098033147c/taskcluster/ci/build/linux.yml#l5.

This causes linux64/opt builds to be built on autoland, even though they look like they shouldn't when you look at the build configuration.

Plus optimization is a whole other beast that is an order of magnitude more complex than run-on-projects. For the optimization portion I filed bug 1641283 and am in the process of a patch that should clarify why each task was either removed or kept.

There was a bunch of discussion about this in #firefox-ci on matrix about this. I don't have a good summary of that discussion.

One point I made is that this may make sense for builds vs. tests, but I don't think it applies to all kinds. In particular, I don't think it applies to any cached-tasks, that are optimized by digest. Most of them have run-on-projects: [] and some have run-on-projects: [trunk] because they are depended on by mach bootstrap which is outside of the graph. It makes sense to pull in toolchains based on dependencies, rather than needing to specify which projects they run on explicitly, if there are not out-of-graph consumers.

A good way of solving this might be to put it in a new transform that asserts run-on-projects is a subset of those in the depedencies and fails the taskgraph otherwise. Then kinds that should have this behaviour can add this transform file to their list.

I'm going to look at this in the next week or two.

I assigned this to bug to myself in error - I won't be looking at it.

There's a key called if-dependencies here that works at the optimization phase, and can be used to only run a task if the dependency would have run anyway. Imo, this key should be used over any sort of check with run-on-projects. More details here:
https://taskcluster-taskgraph.readthedocs.io/en/latest/concepts/task-graphs.html#if-dependencies

When I first implemented if-dependencies I recall I added it to a bunch of the multi_dep follow-up tasks, so this may be largely done. Otherwise, I think we can file new bugs for any specific incidents of this going forward.