Once the Gecko version number is bumped to 69 on mozilla-central, we can drop all Fennec test suites (NOT GeckoView) to Tier 2 and lower run frequency. Per the ESR migration plan, we need to ensure that they remain functional until the beginning of the Gecko 71 cycle, but we will no longer need per-push coverage.

:egao, want to take this one as well :)

The affected tests are the android-common-tests and android-opt-tests:

https://searchfox.org/mozilla-central/rev/a887c90ea9c19a0b5529a1f5fa351929944887ba/taskcluster/ci/test/test-sets.yml#432-446

Or anything on android-em-4.3-arm7-api-16 ("Android 4.3 API16+" on treeherder).

I believe we want to continue all the Android builds per-push, in support of geckoview.

I think builds are ok for now, lets reduce tests to tier-2 and some can be turned off if they are running on android 7.0 as tier-1.

for tests that we are not planning to port (robocop, etc.) I would vote for leaving them for now- when more tests are ported to android 7.0, we can revisit the remaining 4.3 and see where we are.

(In reply to Joel Maher ( :jmaher ) (UTC-4) from comment #4)

I think builds are ok for now, lets reduce tests to tier-2 and some can be turned off if they are running on android 7.0 as tier-1.

Per our previous discussion, we don't want to turn any suites off at this point in time. Just demote to Tier 2 with reduced frequency. We need to ensure that 4.3 tests remain not-broken until the start of the Gecko 71 cycle.

ok, I was remembering incorrectly- thanks for correcting :ryanVM

:ryanvm, do we need pgo and debug, or just pgo?

We should keep both IMO.

Also, I see a number of test suites being demoted here which don't we don't have running as Tier 1 on newer platforms (like reftests, mochitest-plain, etc). We need to be careful about effectively reducing our overall mobile test coverage here. For example, I can't imagine our Mobile team being very happy if the outcome here is that we could break layout on GV and not revert the change because the only failing test suite happens to be Tier 2 running in a 4.3 emulator. The intent of this work is cutting down on redundant suites, not just demoting everything-4.3.

I've updated the original patch (which was a temporary WiP version) with something that should work as intended.

Backed out changeset 22fad4a26360 (bug 1548659) for Decision Task bustage CLOSED TREE

Backout revision https://hg.mozilla.org/integration/autoland/rev/4488934be4ddbbd72fcdc0f5968e596494ab5447

Failed push https://treeherder.mozilla.org/#/jobs?repo=autoland&resultStatus=testfailed%2Cbusted%2Cexception&classifiedState=unclassified&selectedJob=248137844&revision=22fad4a263600854248defec334e89d80c325f3d

Failure log https://treeherder.mozilla.org/logviewer.html#?job_id=248137844&repo=autoland

Edwin can you please take a look?

:arthur_iakab - I identified the issue with the patch, it was a wrong keyword being used for default value.

I have pushed a new try run to verify that decision task is not busted, if that passes I will re-attempt a landing.

https://hg.mozilla.org/mozilla-central/rev/7717ce982073