With some tests running under Fission as tier 1, test-verify should also run for this configuration.

Hi Joel, who should I talk to about enabling more test suites on Treeherder for Fission? This bug in particular is about test-verify.

Tracking for Fission M7 because we'd like all tests to be running before our Fission Beta experiment.

:cpeterson, I would be happy to discuss this and help make things happen. We should do a quick assessment of what is running and what will be running soon as tier-1, and more specifically on what platforms/configurations.

from a quick look, I see mochitest/web-platform-tests as running as tier-1 for linux and windows10, so wpt needs some code added here:
https://searchfox.org/mozilla-central/source/taskcluster/ci/test/web-platform.yml#306

and mochitest needs it added here:
https://searchfox.org/mozilla-central/source/taskcluster/ci/test/misc.yml#44

what we would need to add is something similar to:

    variants:
        by-test-platform:
            linux.*64(-shippable)?-qr/opt: ['fission']
            windows10-64(-shippable)?-qr/opt: ['fission']
            default: []
    fission-tier:
        by-test-platform:
            linux.*64(-shippable)?-qr/opt: 1
            windows10-64(-shippable)?-qr/opt: 1
            default: 2

The above should simplify things- first I want to understand what configs we want to run on and if there are other suites coming online.

Our testing strategy and list of tests we'd like to enable on which platforms:

https://docs.google.com/document/d/19MergQfMob2mdb_s7hlz5Ilj7Bn-Bbi324r6d2bZfxw/edit

We want to run these tests on:

• debug and opt builds
• autoland only (because test-verify is only run on autoland)
• We prefer to run tests with Fission + WebRender (“qr”) where available.

Add platforms:

linux1804-64-qr/debug
linux1804-64-qr/opt
macosx1014-64-qr/debug
macosx1014-64-qr/opt
windows10-64-qr/debug
windows10-64-qr/opt

Andrew, can we get these platforms enabled for TV runs, please?

Sure!

For whatever reason, test-verify only runs with -qr on macosx1014-64. So the fission test-verify jobs will be non-qr on Linux and Windows until we fix that.

This patch will enable the following tasks on autoland:

test-linux1804-64-qr/debug-test-verify-fis-e10s-1
test-linux1804-64-qr/debug-test-verify-fis-e10s-2
test-linux1804-64-qr/debug-test-verify-fis-e10s-3
test-linux1804-64-qr/opt-test-verify-fis-e10s-1
test-linux1804-64-qr/opt-test-verify-fis-e10s-2
test-linux1804-64-qr/opt-test-verify-fis-e10s-3
test-macosx1014-64-qr/debug-test-verify-fis-e10s-1
test-macosx1014-64-qr/debug-test-verify-fis-e10s-2
test-macosx1014-64-qr/debug-test-verify-fis-e10s-3
test-macosx1014-64-qr/opt-test-verify-fis-e10s-1
test-macosx1014-64-qr/opt-test-verify-fis-e10s-2
test-macosx1014-64-qr/opt-test-verify-fis-e10s-3
test-windows10-64-qr/debug-test-verify-fis-e10s-1
test-windows10-64-qr/debug-test-verify-fis-e10s-2
test-windows10-64-qr/debug-test-verify-fis-e10s-3
test-windows10-64-qr/opt-test-verify-fis-e10s-1
test-windows10-64-qr/opt-test-verify-fis-e10s-2
test-windows10-64-qr/opt-test-verify-fis-e10s-3

As you can see from the context above, I went ahead and enabled TV-qr on Linux / Windows so we could run the -fis tasks there. This patch only deals with test-verify though. Chris, do you also want test-verify-gpu and test-verify-wpt? If so, I'll file a new bug for them and add it to our backlog.

(In reply to Andrew Halberstadt [:ahal] from comment #9)

As you can see from the context above, I went ahead and enabled TV-qr on Linux / Windows so we could run the -fis tasks there.

Thanks! That is good. TV with qr is what we want.

This patch only deals with test-verify though. Chris, do you also want test-verify-gpu and test-verify-wpt? If so, I'll file a new bug for them and add it to our backlog.

I don't think we need to bother with test-verify-gpu or test-verify-wpt for Fission.

https://hg.mozilla.org/mozilla-central/rev/b74deabd2aef