James Cheng[:JamesCheng] Reporter
	Description • 8 years ago

I didn't know much about the criteria for enabling WPT test...

Is it possible to enable it on Android Fennec or there is any limitation to enable it on Fennec?

Ideally, we can enable it on try.

Please help me to loop the related people to discuss if it is worth doing this.

	Maja Frydrychowicz :impossibus (was :maja_zf) (needinfo me) Assignee
	Comment 1 • 6 years ago

I'm actually working on this now. I will start by getting wpt running on opt x86 Fennec builds.

	Geoff Brown [:gbrown]
	Comment 2 • 6 years ago

That's great, thanks! 

For local runs, Fennec opt x86 should be fine, when run against a x86 device or emulator, but be aware that the "Android 4.2 x86 opt" platform on treeherder can be unstable: That x86 emulator configuration on aws crashes or hangs frequently for non-trivial jobs. We've never been able to reliably run mochitest-plain or reftests there, for instance.

We are currently trying to stand up a more modern x86 emulator environment based on Android 7.0, with kvm acceleration on packet.net; when that's ready, that will probably be the best place to run wpt.

	Geoff Brown [:gbrown]
	Comment 3 • 6 years ago

Also, are we sure we want to run these against Fennec? We hope to soon be running mochitest and reftest in the geckoview TestRunnerActivity. :snorp might be able to clarify if this hasn't been discussed.

	James Willcox (:snorp) (jwillcox@mozilla.com) (he/him)
	Comment 4 • 6 years ago

Yeah I agree I think we want to run these in TestRunnerActivity and not Fennec.

	Maja Frydrychowicz :impossibus (was :maja_zf) (needinfo me) Assignee
	Comment 5 • 6 years ago

Awesome, thanks for the extra info. 

Re TestRunnerActivity, my impression that, yes, ultimately the goal is to run wpt against geckoview, but running wpt on Fennec is (hopefully) a simple intermediate step in that direction that leverages what's already in place for Marionette. I'll discuss this with my team and post updates.

	Maja Frydrychowicz :impossibus (was :maja_zf) (needinfo me) Assignee
	Comment 6 • 6 years ago

How ready is the geckoview TestRunnerActivity? One reason I've started working on wpt for Fennec now is because we were under the impression that geckoview needs more work.

Geoff -- what's the bug for the mochitest/reftest work for TestRunnerActivity?

	James Graham [:jgraham]
	Comment 7 • 6 years ago

What's the remote-control situation for TestRunnerActivity? Will it have marionette?

	Geoff Brown [:gbrown]
	Comment 8 • 6 years ago

(In reply to Maja Frydrychowicz (:maja_zf) from comment #6)
> Geoff -- what's the bug for the mochitest/reftest work for
> TestRunnerActivity?

Bug 1460411, just filed. I'll try to demonstrate current support in a try push, referenced in that bug...soon.

	Maja Frydrychowicz :impossibus (was :maja_zf) (needinfo me) Assignee
	Comment 9 • 6 years ago

(In reply to James Graham [:jgraham] from comment #7)
> What's the remote-control situation for TestRunnerActivity? Will it have
> marionette?

Looks like this was looked into in the past, but we'll need to revisit the question: https://bugzilla.mozilla.org/show_bug.cgi?id=1411658#c12

	Maja Frydrychowicz :impossibus (was :maja_zf) (needinfo me) Assignee
	Comment 10 • 6 years ago

Update: we decided a few weeks ago to add support for geckoview to Marionette, and that way we can run wpt against a geckoview "product" in wpt parlance that would likely represent the geckoview TestRunnerActivity.

Since Marionette already works on Fennec, I decided to hook up wptrunner to a new "fennec" product, which is what I'll post as WIP patch in a moment. Much of the same work would apply to a creatng geckoview-based product from the wptrunner end of things.

	Phabricator Automation
	Comment 11 • 6 years ago

This allows running web-platform-tests on Fennec given a running emulator.
(Which is how we expect the tests to run in automation as well -- the
android_emulator_unittest mozharness script takes care of emulator
start-up.) It also hooks up ./mach wpt.

wptrunner sets up a profile for Fennec, forwards the marionette port
and starts up Fennec, etc.

Some config is hard-coded just to get it working. Also, the
tests navigate to the host system's ip instead of web-platform.test
URL (TODO - proxy autoconfig?).

= Usage =

Set your mozconfig to build fennec for x86 (I only tested on x86
because I can't run the ARM emulator locally for some reason.)

Start an emulator: `./mach android-emulator --version x86`
Install fennec: `./mach build && ./mach package && ./mach install`
Run the tests: `./mach wpt --product=fennec --testtype=testharness
--certutil-binary path/to/host/os/certutil path/to/some/tests`

	Maja Frydrychowicz :impossibus (was :maja_zf) (needinfo me) Assignee
	Comment 14 • 6 years ago

I'm not requesting review yet, just sharing the progress so far.

	Phabricator Automation
	Comment 15 • 6 years ago

Comment on attachment 8984313 [details]
Bug 1323620 - Add example of running Fennec to Marionette docs.

James Graham [:jgraham] has approved the revision.

https://phabricator.services.mozilla.com/D1590

	Maja Frydrychowicz :impossibus (was :maja_zf) (needinfo me) Assignee
	Comment 16 • 6 years ago

https://treeherder.mozilla.org/#/jobs?repo=try&revision=d2d99b48e61f6e264d11a08adea320c78a010743

	Phabricator Automation
	Comment 17 • 6 years ago

Comment on attachment 8984312 [details]
Bug 1323620 - (TEMP) Copy FirefoxBrowser - separate commit for easier review

James Graham [:jgraham] has approved the revision.

https://phabricator.services.mozilla.com/D1589

	Phabricator Automation
	Comment 18 • 6 years ago

Comment on attachment 8984309 [details]
Bug 1323620 - Add "fennec" product to wptrunner.

James Graham [:jgraham] has approved the revision.

https://phabricator.services.mozilla.com/D1587

	Pulsebot
	Comment 19 • 6 years ago

Pushed by mjzffr@gmail.com:
https://hg.mozilla.org/integration/autoland/rev/c1d664fefdff
Add example of running Fennec to Marionette docs. r=jgraham

	Pulsebot
	Comment 20 • 6 years ago

Pushed by mjzffr@gmail.com:
https://hg.mozilla.org/integration/autoland/rev/266c6cfb96a7
Add "fennec" product to wptrunner. r=jgraham

	Web Platform Test Sync Bot (Matrix: #interop:mozilla.org)
	Comment 21 • 6 years ago

Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/11530 for changes under testing/web-platform/tests

	Web Platform Test Sync Bot (Matrix: #interop:mozilla.org)
	Comment 22 • 6 years ago

Can't merge web-platform-tests PR due to failing upstream checks:
Github PR https://github.com/web-platform-tests/wpt/pull/11530
* continuous-integration/travis-ci/pr (https://travis-ci.org/web-platform-tests/wpt/builds/392795868?utm_source=github_status&utm_medium=notification)

	Phabricator Automation
	Comment 24 • 6 years ago

Comment on attachment 8985673 [details]
Bug 1323620 - Remove unused imports

James Graham [:jgraham] has approved the revision.

https://phabricator.services.mozilla.com/D1670

	Pulsebot
	Comment 25 • 6 years ago

Pushed by mjzffr@gmail.com:
https://hg.mozilla.org/integration/autoland/rev/31a705addba6
Remove unused imports r=jgraham

	Web Platform Test Sync Bot (Matrix: #interop:mozilla.org)
	Comment 26 • 6 years ago

Can't merge web-platform-tests PR due to failing upstream checks:
Github PR https://github.com/web-platform-tests/wpt/pull/11530
* continuous-integration/travis-ci/pr (https://travis-ci.org/web-platform-tests/wpt/builds/392835317?utm_source=github_status&utm_medium=notification)

	Arthur Iakab [arthur_iakab]
	Comment 27 • 6 years ago
bugherder

https://hg.mozilla.org/mozilla-central/rev/c1d664fefdff
https://hg.mozilla.org/mozilla-central/rev/266c6cfb96a7
https://hg.mozilla.org/mozilla-central/rev/31a705addba6

	Phabricator Automation
	Comment 29 • 6 years ago

Comment on attachment 8985725 [details]
Bug 1323620 - Fix test_products.py

James Graham [:jgraham] has approved the revision.

https://phabricator.services.mozilla.com/D1677

	Pulsebot
	Comment 30 • 6 years ago

Pushed by mjzffr@gmail.com:
https://hg.mozilla.org/integration/autoland/rev/c792f7662614
Fix test_products.py r=jgraham

	Web Platform Test Sync Bot (Matrix: #interop:mozilla.org)
	Comment 31 • 6 years ago

Can't merge web-platform-tests PR due to failing upstream checks:
Github PR https://github.com/web-platform-tests/wpt/pull/11530
* continuous-integration/travis-ci/pr (https://travis-ci.org/web-platform-tests/wpt/builds/392962896?utm_source=github_status&utm_medium=notification)

	Cristian Brindusan [:cbrindusan]
	Comment 32 • 6 years ago
bugherder

https://hg.mozilla.org/mozilla-central/rev/c792f7662614

	Web Platform Test Sync Bot (Matrix: #interop:mozilla.org)
	Comment 33 • 6 years ago

Upstream PR merged