Bug 1657769 started running a number of Talos tests with software WebRender enabled, however there's no way in Perfherder to distinguish these results from the 'regular' WebRender results.

In Raptor we're adding 'webrender' to extraOptions as a way to distinguish these results (in addition to the -qr platform suffix). Perhaps we should also include this in Talos, with a variant of 'webrender-sw'? I'm open to other suggestions, but I am keen to avoid overloading the test name with these characteristics.

Example results in Perfherder: https://treeherder.mozilla.org/perf.html#/graphs?highlightAlerts=1&series=autoland,1925804,1,1&timerange=5184000

It looks like adding a 'webrender' extra option should be simple based on what we're doing for Fission:

# differentiate fission vs non-fission results in perfherder
if browser_config.get('enable_fission', False):
    talos_results.add_extra_option('fission')

We should be able to use enable_webrender for 'regular webrender, but I'm less sure how we can detect software enabled WebRender. :ahal do you have any suggestions? Can we check the gfx.webrender.software pref, or the presence of this on the command line?

Yes, here's the webrender-sw definition:
https://searchfox.org/mozilla-central/source/taskcluster/taskgraph/transforms/tests.py#256

So either that pref, or you could check for -swr in the label, or we could add a tag or something to the tags attribute. Personally I feel like checking for the pref is the safest because then even if a developer manually uses --setpref gfx.webrender.software=1 on the command line, then webrender-sw will still be detected.

Note, we do similar things in the mochitest harness:
https://searchfox.org/mozilla-central/source/testing/mochitest/runtests.py#2671

I believe T-gli is affected by this issue too.

(In reply to Matt Woodrow (:mattwoodrow) from comment #4)

I believe T-gli is affected by this issue too.

As far as I can tell, the tests in T-gli are unique to that group so I don't think we're mixing these results with others.

(In reply to Dave Hunt [:davehunt] [he/him] ⌚BST from comment #5)

(In reply to Matt Woodrow (:mattwoodrow) from comment #4)

I believe T-gli is affected by this issue too.

As far as I can tell, the tests in T-gli are unique to that group so I don't think we're mixing these results with others.

OSX 10.14 Shippable opt - T(webgl) and T-gli(webgl) had combined results on my try run. The numbers are pretty similar though, so it's not obviously bimodal.

(In reply to Matt Woodrow (:mattwoodrow) from comment #6)

(In reply to Dave Hunt [:davehunt] [he/him] ⌚BST from comment #5)

(In reply to Matt Woodrow (:mattwoodrow) from comment #4)

I believe T-gli is affected by this issue too.

As far as I can tell, the tests in T-gli are unique to that group so I don't think we're mixing these results with others.

OSX 10.14 Shippable opt - T(webgl) and T-gli(webgl) had combined results on my try run. The numbers are pretty similar though, so it's not obviously bimodal.

You're correct, here's a mix for T(webgl) and T-gli(webgl). I'll clone this bug, and we should add an extra option to distinguish webgl-ipc.

Set release status flags based on info from the regressing bug 1657769

This patch adds extra options to the perfherder data for webrender, webrender-sw (webrender-software), and webgl-ipc (to distinguish from webgl results).

https://hg.mozilla.org/mozilla-central/rev/ce2c3809f07f