We're looking to ship software webrender in the near future, so want to start running reftests.

Ideally these would be the same as the "Linux 18.04 x64 WebRender debug" tests, except with the "gfx.webrender.software=true" pref added.

Unfortunately these aren't green just yet, so we might need them to be hidden to begin with, or wait until they are fully green.

Now that 1656813 has landed, these should be green.

Andrew, could you please help with getting this new test configuration added?

:mattwoodrow - Plain reftests only, or crashtests too? jsreftests??

Probably just plain reftests is fine for now. Thank you!

It looks like there is just one perma-fail then, dom/canvas/test/reftest/webgl-color-test.html:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=3b8a8c00552566a686329e3d1ab026c3d967f90f

If we can resolve that I can go ahead with this as tier 1; is that an option?

That sounds great!

I think we should just mark that test as fails-if(swgl) for now, and we can resolve it as a separate task.

Filed bug 1657771 to fix the canvas issue, will write a patch to mark it as failing.

Matt, thanks for all your help here.

One last question...
As seen here, the new reftests run alongside the original ones for the "Linux 18.04 x64 WebRender debug" test platform:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=e2fa9aa57ae2f8b73698dbc026b1ce6c4f848f28

What is your preference for the new group suffix:
R-sw ("software" - webrender already in test platform)
R-swr ("software webrender")
R-swgl (like in the reftest sandbox)
R-(something else)?

R-swr sounds great to me.

https://hg.mozilla.org/mozilla-central/rev/910fdb11c7a1
https://hg.mozilla.org/mozilla-central/rev/d4f23cd40606