We currently run the SETA algorithm as a shadow scheduler, so that we can compare the algorithms we're implementing against it.

However the SETA algorithm has a baked in "backstop" that runs every task every 10th push. We do this so we guarantee we don't miss any failing tasks and so sheriffs have a good merge candidate.

The problem is that we'll want to do something similar with all of the other algorithms as well. And right now, the SETA results look worse than they are as they are scheduling way more tasks because of the backstop, so the comparison is not fair.

Since we will want to share this backstop across algorithms, I propose we pull it out of SETA and into its own OptimizationStrategy. So the strategy we use would look something like skip-unless-schedules-or-seta-or-backstop.

I'd like to implement a 'backstop' strategy, such that it will prevent all other
optimizers from removing tasks under certain conditions (e.g every 10th push).

The nicest way to implement this seems to be an 'All' composite strategy
(similar to 'Either' which this patch renames to 'Any'). This means we could
do something like:

All("seta", "backstop")

which means we would only remove tasks if all substrategies say to remove
tasks.

This allows to nest strategies without having to register ever intermediate
composite strategy first. For example:

All(Any("skip-unless-schedules", "seta"), "backstop")

Prior to this patch, we'd need to register that 'Any' one first and then use it
in the 'All'.

Depends on D68620

We'll want some kind of backstop no matter what optimization algorithm we use.
We don't want to go too long without running any given task so we can find
regressions quickly and have a good merge candidate.

This pulls the logic that handles this out of the SETA strategy and into its
own strategy.

This will also make the SETA shadow scheduler more representative of what the
algorithm is doing.

Note in the future we may find ways to make this backstop more efficient (i.e
only run tasks that didn't run in the last 9 pushes for example).

Depends on D68621

Backed out for breaking gecko decision task

Push with failure: https://treeherder.mozilla.org/#/jobs?repo=autoland&resultStatus=testfailed%2Cbusted%2Cexception&revision=efffde76e782010f123b39b888be223679ab86bb&selectedJob=297767751

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=297767751&repo=autoland&lineNumber=53

Backout: https://hg.mozilla.org/integration/autoland/rev/63681622a50cdd91b643fdeec1e93b1855cfae64

I could have sworn that I caught this conflict, fixed it and resubmitted it to Phabricator.. but I guess the phabricator diff doesn't lie and I either forgot the last step or the command failed and I didn't notice.

https://hg.mozilla.org/mozilla-central/rev/cd8200c37dd3
https://hg.mozilla.org/mozilla-central/rev/a81b9c7a4324
https://hg.mozilla.org/mozilla-central/rev/913568b1ee34