I didn't know much about the criteria for enabling WPT test... Is it possible to enable it on Android Fennec or there is any limitation to enable it on Fennec? Ideally, we can enable it on try. Please help me to loop the related people to discuss if it is worth doing this.

I'm actually working on this now. I will start by getting wpt running on opt x86 Fennec builds.

That's great, thanks! For local runs, Fennec opt x86 should be fine, when run against a x86 device or emulator, but be aware that the "Android 4.2 x86 opt" platform on treeherder can be unstable: That x86 emulator configuration on aws crashes or hangs frequently for non-trivial jobs. We've never been able to reliably run mochitest-plain or reftests there, for instance. We are currently trying to stand up a more modern x86 emulator environment based on Android 7.0, with kvm acceleration on packet.net; when that's ready, that will probably be the best place to run wpt.

Also, are we sure we want to run these against Fennec? We hope to soon be running mochitest and reftest in the geckoview TestRunnerActivity. :snorp might be able to clarify if this hasn't been discussed.

Yeah I agree I think we want to run these in TestRunnerActivity and not Fennec.

Awesome, thanks for the extra info. Re TestRunnerActivity, my impression that, yes, ultimately the goal is to run wpt against geckoview, but running wpt on Fennec is (hopefully) a simple intermediate step in that direction that leverages what's already in place for Marionette. I'll discuss this with my team and post updates.

How ready is the geckoview TestRunnerActivity? One reason I've started working on wpt for Fennec now is because we were under the impression that geckoview needs more work. Geoff -- what's the bug for the mochitest/reftest work for TestRunnerActivity?

What's the remote-control situation for TestRunnerActivity? Will it have marionette?

(In reply to Maja Frydrychowicz (:maja_zf) from comment #6) > Geoff -- what's the bug for the mochitest/reftest work for > TestRunnerActivity? Bug 1460411, just filed. I'll try to demonstrate current support in a try push, referenced in that bug...soon.

(In reply to James Graham [:jgraham] from comment #7) > What's the remote-control situation for TestRunnerActivity? Will it have > marionette? Looks like this was looked into in the past, but we'll need to revisit the question: https://bugzilla.mozilla.org/show_bug.cgi?id=1411658#c12

Update: we decided a few weeks ago to add support for geckoview to Marionette, and that way we can run wpt against a geckoview "product" in wpt parlance that would likely represent the geckoview TestRunnerActivity. Since Marionette already works on Fennec, I decided to hook up wptrunner to a new "fennec" product, which is what I'll post as WIP patch in a moment. Much of the same work would apply to a creatng geckoview-based product from the wptrunner end of things.

This allows running web-platform-tests on Fennec given a running emulator. (Which is how we expect the tests to run in automation as well -- the android_emulator_unittest mozharness script takes care of emulator start-up.) It also hooks up ./mach wpt. wptrunner sets up a profile for Fennec, forwards the marionette port and starts up Fennec, etc. Some config is hard-coded just to get it working. Also, the tests navigate to the host system's ip instead of web-platform.test URL (TODO - proxy autoconfig?). = Usage = Set your mozconfig to build fennec for x86 (I only tested on x86 because I can't run the ARM emulator locally for some reason.) Start an emulator: `./mach android-emulator --version x86` Install fennec: `./mach build && ./mach package && ./mach install` Run the tests: `./mach wpt --product=fennec --testtype=testharness --certutil-binary path/to/host/os/certutil path/to/some/tests`

I'm not requesting review yet, just sharing the progress so far.

Comment on attachment 8984313 [details] Bug 1323620 - Add example of running Fennec to Marionette docs. James Graham [:jgraham] has approved the revision. https://phabricator.services.mozilla.com/D1590

Comment on attachment 8984312 [details] Bug 1323620 - (TEMP) Copy FirefoxBrowser - separate commit for easier review James Graham [:jgraham] has approved the revision. https://phabricator.services.mozilla.com/D1589

Comment on attachment 8984309 [details] Bug 1323620 - Add "fennec" product to wptrunner. James Graham [:jgraham] has approved the revision. https://phabricator.services.mozilla.com/D1587

Pushed by mjzffr@gmail.com: https://hg.mozilla.org/integration/autoland/rev/c1d664fefdff Add example of running Fennec to Marionette docs. r=jgraham

Pushed by mjzffr@gmail.com: https://hg.mozilla.org/integration/autoland/rev/266c6cfb96a7 Add "fennec" product to wptrunner. r=jgraham

Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/11530 for changes under testing/web-platform/tests

Can't merge web-platform-tests PR due to failing upstream checks: Github PR https://github.com/web-platform-tests/wpt/pull/11530 * continuous-integration/travis-ci/pr (https://travis-ci.org/web-platform-tests/wpt/builds/392795868?utm_source=github_status&utm_medium=notification)

Comment on attachment 8985673 [details] Bug 1323620 - Remove unused imports James Graham [:jgraham] has approved the revision. https://phabricator.services.mozilla.com/D1670

Pushed by mjzffr@gmail.com: https://hg.mozilla.org/integration/autoland/rev/31a705addba6 Remove unused imports r=jgraham

Can't merge web-platform-tests PR due to failing upstream checks: Github PR https://github.com/web-platform-tests/wpt/pull/11530 * continuous-integration/travis-ci/pr (https://travis-ci.org/web-platform-tests/wpt/builds/392835317?utm_source=github_status&utm_medium=notification)

https://hg.mozilla.org/mozilla-central/rev/c1d664fefdff https://hg.mozilla.org/mozilla-central/rev/266c6cfb96a7 https://hg.mozilla.org/mozilla-central/rev/31a705addba6

Comment on attachment 8985725 [details] Bug 1323620 - Fix test_products.py James Graham [:jgraham] has approved the revision. https://phabricator.services.mozilla.com/D1677

Pushed by mjzffr@gmail.com: https://hg.mozilla.org/integration/autoland/rev/c792f7662614 Fix test_products.py r=jgraham

Can't merge web-platform-tests PR due to failing upstream checks: Github PR https://github.com/web-platform-tests/wpt/pull/11530 * continuous-integration/travis-ci/pr (https://travis-ci.org/web-platform-tests/wpt/builds/392962896?utm_source=github_status&utm_medium=notification)

Upstream PR merged