Closed
Bug 797164
Opened 12 years ago
Closed 10 years ago
Add support for mochitest-chrome to B2G
Categories
(Testing :: Mochitest, defect, P1)
Tracking
(blocking-basecamp:+)
RESOLVED
FIXED
blocking-basecamp | + |
People
(Reporter: jgriffin, Assigned: jgriffin)
References
Details
(Whiteboard: [LOE:M])
Attachments
(3 files, 6 obsolete files)
(deleted),
patch
|
ahal
:
review+
|
Details | Diff | Splinter Review |
(deleted),
patch
|
ahal
:
review+
|
Details | Diff | Splinter Review |
(deleted),
patch
|
ahal
:
review+
|
Details | Diff | Splinter Review |
Looks like we'll need to add support for mochitest-chrome for B2G.
Blocks: 780955
Comment 1•12 years ago
|
||
Noming since this blocks getting the existing updater mochitests running for B2G
blocking-basecamp: --- → ?
Comment 2•12 years ago
|
||
Blocking all these suites is important enough to block on IMHO.
blocking-basecamp: ? → +
Comment 3•12 years ago
|
||
jgriffin: I'm potentially interested in helping here. Can you provide more details? ping on IRC if needed.
Flags: needinfo?(jgriffin)
Assignee | ||
Comment 4•12 years ago
|
||
(In reply to Gregory Szorc [:gps] from comment #3)
> jgriffin: I'm potentially interested in helping here. Can you provide more
> details? ping on IRC if needed.
mochitest-plain on B2G works by using Marionette to navigate to the mochitest test page here: http://mxr.mozilla.org/mozilla-central/source/build/mobile/b2gautomation.py#271
I imagine for mochitest chrome, we'd do something similar, except that instead of loading the test page into the existing content window, we'd load it in place of shell.xul. This probably would look like this:
self.marionette.set_context(self.marionette.CONTEXT_CHROME)
self.marionette.navigate(self.testURL)
Performing the navigation in the 'chrome' context causes us to replace the primary XUL page that's currently loaded, which in the case of B2G is shell.xul.
Btw, mochitest-plain is currently broken on B2G due to bug 798580. This probably won't impact efforts to get mochitest-chrome working, AFAICT.
Flags: needinfo?(jgriffin)
Assignee | ||
Comment 5•12 years ago
|
||
It turns out, of course, that things aren't quite so simple. I've been looking into this and am running into this error:
E/GeckoConsole( 429): [JavaScript Error: "TypeError: linkArray is undefined" {file: "chrome://mochikit/content/server.js" line: 534}]
Investigation continues.
Assignee | ||
Comment 6•12 years ago
|
||
The problem is that when the chrome harness tries to build a test list, it takes the url chrome://mochitests/content/mochitests.xul and resolves it into a local file url. However, the way it does this (http://mxr.mozilla.org/mozilla-central/source/testing/mochitest/chrome-harness.js#32) resolves to a file url on the machine used to compile the build, not the device (e.g., in my case file:///home/jgriffin/armB2G/objdir-gecko/_tests/testing/mochitest/mochitests.xul).
I'd been working under the assumption that mochitest-chrome was run at some point in the past on mobile, but maybe I'm wrong. Do you know Joel? If it did, I'm not sure how it worked around this. If it didn't, then we'll have to adapt this somehow for the remote case.
Either way it's more work than I thought.
Comment 7•12 years ago
|
||
mochitest-chrome has never run on android. We did run it on n900, but that was a local machine. Likewise mochitest-browser-chrome has never run remotely.
Is there a need for mochitest-chrome? iirc, many of the tests are 'Firefox' specific as they have been rolled into that suite for higher privileges.
Assignee | ||
Comment 8•12 years ago
|
||
(In reply to Joel Maher (:jmaher) from comment #7)
> mochitest-chrome has never run on android. We did run it on n900, but that
> was a local machine. Likewise mochitest-browser-chrome has never run
> remotely.
>
> Is there a need for mochitest-chrome? iirc, many of the tests are 'Firefox'
> specific as they have been rolled into that suite for higher privileges.
Unfortunately, there is a need. QA assures me that some app permissions tests cannot be run in mochitest-plain, and marshall_law wants mochitest-chrome support for update tests.
Comment 9•12 years ago
|
||
I would suggest copying the test files to the device and we could hack the harness to make it work with a different root directory.
Comment 10•12 years ago
|
||
I thought all the work you did to run mochitest-chrome from a jar file (bug 543800) was in support of running it on mobile? In any event, that's how they're run nowadays, so it should just be a matter of getting the jar file installed to the profile and maybe some fiddling with the harness. The tests all run from chrome: URLs which map back to jar URLs.
Comment 11•12 years ago
|
||
All P1s need an assignee. Please re-assign or change the LOE as appropriate, jgriffin.
Assignee: nobody → jgriffin
Priority: -- → P1
Whiteboard: [LOE:M]
Assignee | ||
Comment 12•12 years ago
|
||
Ah, I see. The chrome JAR file only gets made during 'make package-tests', so it isn't possible to run remote chrome tests via 'make mochitest-chrome', or directly via the runner in the objdir/_tests/testing/mochitest dir.
Comment 13•12 years ago
|
||
you could generate the .jar file from 'make mochitest-chrome' and copy it to the device/emulator without much trouble.
Assignee | ||
Comment 14•12 years ago
|
||
It appears there are no longer any use cases requiring this. If one appears, we'll re-open this.
Status: NEW → RESOLVED
Closed: 12 years ago
Resolution: --- → WONTFIX
Comment 15•10 years ago
|
||
We have a use case now. See bug 1021060 comment 16 and onward.
Status: RESOLVED → REOPENED
Flags: needinfo?(jgriffin)
Resolution: WONTFIX → ---
Assignee | ||
Comment 16•10 years ago
|
||
Bug 1021060 makes it sound as if mochitest-chrome on Android is a higher priority than on B2G. I'll plan for this in early Q3, but we won't be able to implement in the next couple of weeks.
Flags: needinfo?(jgriffin)
Comment 17•10 years ago
|
||
That's because requestAutocomplete is being implemented for desktop and android at the moment. Bug 964295 tracks a Firefox OS UI but I don't know of the timeline on that. Does a separate bug need to be filed for Android?
Perhaps there should be a newsgroup discussion on whether it's worth the infra resources to run chrome tests on every platform. Since we seem to run mochitest-plain everywhere it would seems like we think it is worth it since they aren't that different in terms of what they're testing.
Comment 18•10 years ago
|
||
I would question the value of mochitest-chrome on android. I suspect about half of the test cases will not work due to lack of features (note: different UI). Is this worth the effort? Likewise will we have many chrome cases for b2g?
Comment 19•10 years ago
|
||
Half is a lot of tests though. I suspect a lot of the chrome tests in dom/, content/, and some in toolkit/ aren't related to the UI and could run on mobile. I'm not sure how common it would be for these tests to fail on mobile but not desktop.
Assignee | ||
Comment 20•10 years ago
|
||
The issue, I think, is that some tests need to execute some setup code in chrome before they do something in content. SpecialPowers works ok in mochitest-plain for some cases, but the more complex the chrome setup code required, the harder it is to use SpecialPowers.
Yes, we should file a separate bug for mochitest-chrome on Android, since that's a completely different code path, harness-wise. And yes, I think a newsgroup thread is appropriate; I volunteer to start one tomorrow.
Comment 21•10 years ago
|
||
please be clear that we will be adding harness support for mochitest-chrome to android/b2g instead of turning on mochitest-chrome which indicates the majority of the chrome tests.
Assignee | ||
Comment 22•10 years ago
|
||
(In reply to Joel Maher (:jmaher) from comment #21)
> please be clear that we will be adding harness support for mochitest-chrome
> to android/b2g instead of turning on mochitest-chrome which indicates the
> majority of the chrome tests.
Agreed.
Updated•10 years ago
|
Comment 23•10 years ago
|
||
(In reply to Joel Maher (:jmaher) from comment #18)
> I would question the value of mochitest-chrome on android. I suspect about
> half of the test cases will not work due to lack of features (note:
> different UI).
Aside from tests written before the introduction of mochitest-browser, what kind of mochitest-chrome tests would be depending on UI feature? The point of mochitest-chrome is to run privileged stuff _without_ the dependency on the Firefox browser-chrome (which is why, for example, we can run them on Thunderbird CI).
Assignee | ||
Comment 24•10 years ago
|
||
Most of the mochitest-chrome tests won't work in B2G because they use .xul files, which aren't well-supported in B2G. I don't know what level of support we have for .xul files in Android.
Comment 25•10 years ago
|
||
I would be happy to require/encourage that new mochitest-chrome tests use .html, and write a PSA to dev-platform. Most mochitest-chrome files just use .xul by convention.
Does the standard mochitest-plain template (created via gen_template.pl) work out-of-the-box in mochitest-chrome?
Assignee | ||
Comment 26•10 years ago
|
||
Probably not, chrome tests need chrome:// url's for SimpleTest.js and friends, whereas the mochitest-plain template uses a relative url.
Comment 27•10 years ago
|
||
(In reply to Jonathan Griffin (:jgriffin) from comment #26)
> Probably not, chrome tests need chrome:// url's for SimpleTest.js and
> friends, whereas the mochitest-plain template uses a relative url.
Ok. Can we switch the template generated by gen_template.pl -type chrome to spit out an html file?
Assignee | ||
Comment 28•10 years ago
|
||
Yes, just filed bug 1026785 for this.
Assignee | ||
Comment 29•10 years ago
|
||
Unassigning since I'm not actively working on this. We're in the process of deciding whether we should pick this up for Q3.
Assignee: jgriffin → nobody
Comment 30•10 years ago
|
||
See https://groups.google.com/d/msg/mozilla.dev.platform/rdaX9q33nGc/R-qQbiG2My4J
jgriffin, can we get an update on this?
Flags: needinfo?(jgriffin)
Assignee | ||
Comment 31•10 years ago
|
||
We haven't started yet, and might not be able to until early Q4.
Flags: needinfo?(jgriffin)
Comment 32•10 years ago
|
||
(In reply to Jonathan Griffin (:jgriffin) from comment #31)
> We haven't started yet, and might not be able to until early Q4.
Ok - good to know! As long as it happens sometime in the not-too-distant future, I can feel a lot better about writing mochitest-chrome tests. :-)
Updated•10 years ago
|
Assignee: nobody → gbrown
Comment 33•10 years ago
|
||
I'm treating 1026290 as a dependency -- getting chrome tests working on Android first. I'm making good progress there and hope to apply what I'm learning to b2g very soon now.
Comment 35•10 years ago
|
||
We can run mochitest-chrome on Android now (via runtestsremote.py with --chrome specified); not all tests pass, but all tests can be run! There were 3 issues on Android:
- there was an initialization race sometimes preventing the "contentEvent" from being received
- logFile (and maybe other) parameters were not passed in testConfig.js
- chrome test files were missing from device and tests.manifest was pointing to a non-remote location.
If I address these issues for b2g, mochitest-chrome still fails. On b2g, redirect.html is loaded and sends the contentEvent, but contentEvent is never received:
http://hg.mozilla.org/mozilla-central/annotate/29d086b32a26/testing/mochitest/browser-test.js#l72
In fact, it appears that browser-test.js is never loaded.
browser-test.js is in mochikit.jar and referenced from browser-test-overlay.xul - http://hg.mozilla.org/mozilla-central/annotate/29d086b32a26/testing/mochitest/browser-test-overlay.xul#l10, which is also in mochikit.jar.
mochikit.jar is pushed to device at /data/local/tests/profile/extensions/staged/mochikit@mozilla.org. Unexpectedly, I find that it appears to never be moved from "staged"...is mochikit.jar pushed but somehow not used on b2g?
(Also, :jmaher noted that mochitest-chrome on desktop Firefox behaves very similarly with --e10s: redirect.html is loaded but nothing else happens; we wonder if there is a common cause.)
Comment 36•10 years ago
|
||
Another possible point of failure: browser-test-overlay.xul is referenced from the mochikit chrome.manifest, which would normally contain:
overlay chrome://browser/content/browser.xul chrome://mochikit/content/browser-test-overlay.xul
overlay chrome://browser/content/shell.xhtml chrome://mochikit/content/browser-test-overlay.xul
overlay chrome://navigator/content/navigator.xul chrome://mochikit/content/browser-test-overlay.xul
overlay chrome://webapprt/content/webapp.xul chrome://mochikit/content/browser-test-overlay.xul
I've tried variations such as
overlay chrome://b2g/content/shell.html chrome://mochikit/content/browser-test-overlay.xul
but I'm not certain what this should be.
Comment 37•10 years ago
|
||
mochikit seems to be copied to /system/b2g/distribution/bundles/mochikit@mozilla.org, complete with chrome.manifest and the jar.
Assignee | ||
Comment 38•10 years ago
|
||
For B2G and mochitest-plain, we run the tests in a child process, since we run them inside an out-of-process Gaia app, 'test-container'. In the context of this app, shell.html doesn't exist (it only exists for the top-level context).
> I've tried variations such as
>
> overlay chrome://b2g/content/shell.html chrome://mochikit/content/browser-test-overlay.xul
That looks right, but I'm also not positive that chrome overlays work in B2G. Let's figure out whether we need these tests to run in the parent process or in a child process, and then we can figure out how to get there.
Bobby, should mochitest-chrome for B2G run tests in the top-level process, or in an OOP app?
Flags: needinfo?(bobbyholley)
Comment 39•10 years ago
|
||
They should run in the parent process (so that they have access to the usual non-sandboxed stuff), and they should not be using XUL or overlays at all. They should simply be html-based - just like mochitest-plain, but loaded from a chrome:// URI rather than an http:// URI.
Flags: needinfo?(bobbyholley)
Assignee | ||
Comment 40•10 years ago
|
||
I think what we should do then, gbrown, is make a version of shell.html that we package with the mochitest-browser files that loads the same .js files as browser-test-overlay.xul. We can then use Marionette to navigate to that file.
Comment 41•10 years ago
|
||
Thanks jgriffin.
First I checked to see what would happen if I tried loading those .js files from <script> tags in shell.html itself. That reports:
I/GeckoConsole( 44): No chrome package registered for chrome://mochikit/content/chrome-harness.js
I/GeckoConsole( 44): No chrome package registered for chrome://mochikit/content/browser-test.js
I/GeckoConsole( 44): No chrome package registered for chrome://mochikit/content/cc-analyzer.js
I/GeckoConsole( 44): No chrome package registered for chrome://mochikit/content/mochitest-e10s-utils.js
but appears to successfully load those files and call the onLoad handler in browser-test.js.
I also tried loading from b2g_start_script.js via loadSubScript(). That reports no errors, but the onLoad handler in browser-test.js is not called.
Comment 42•10 years ago
|
||
I found a minor change to browser-test.js was needed (but this does not solve the problem from comment 41):
diff --git a/testing/mochitest/browser-test.js b/testing/mochitest/browser-test.js
index a989d15..e6da999 100644
--- a/testing/mochitest/browser-test.js
+++ b/testing/mochitest/browser-test.js
@@ -5,7 +5,11 @@ var gConfig;
if (Cc === undefined) {
var Cc = Components.classes;
+}
+if (Ci === undefined) {
var Ci = Components.interfaces;
+}
+if (Cu === undefined) {
var Cu = Components.utils;
}
Assignee | ||
Comment 43•10 years ago
|
||
(In reply to Geoff Brown [:gbrown] from comment #41)
> Thanks jgriffin.
>
> First I checked to see what would happen if I tried loading those .js files
> from <script> tags in shell.html itself. That reports:
>
> I/GeckoConsole( 44): No chrome package registered for
> chrome://mochikit/content/chrome-harness.js
> I/GeckoConsole( 44): No chrome package registered for
> chrome://mochikit/content/browser-test.js
> I/GeckoConsole( 44): No chrome package registered for
> chrome://mochikit/content/cc-analyzer.js
> I/GeckoConsole( 44): No chrome package registered for
> chrome://mochikit/content/mochitest-e10s-utils.js
>
> but appears to successfully load those files and call the onLoad handler in
> browser-test.js.
>
> I also tried loading from b2g_start_script.js via loadSubScript(). That
> reports no errors, but the onLoad handler in browser-test.js is not called.
If you load it via loadSubScript(), I think you'll need to then call testInit(); I wouldn't expect the onLoad handler to be triggered.
Comment 44•10 years ago
|
||
After talking with :ahal, I'm calling loadSubScript() from b2g_start_script to load the js normally loaded via browser-test-overlay.xul. Instead of relying on the load and MozAfterPaint events to call testInit(), I'm calling it directly -- that seems to work okay.
In testInit(), we wait for "contentEvent", normally sent from redirect.html; that's never received, presumably because of e10s. I've hacked around that, invoking that with a setTimeout instead.
That allows me to load the harness, and start to parse the test list, but then I start hitting various js errors. All of those problems seem to relate back to SpecialPowers being undefined. I wouldn't expect SpecialPowers to be needed, but it turns out there are various references: http://mxr.mozilla.org/mozilla-central/source/testing/mochitest/tests/SimpleTest/setup.js#105.
I can't quite account for why SpecialPowers is undefined in this environment while it doesn't seem to cause problems anywhere else.
Comment 45•10 years ago
|
||
I don't know if you included the ChromePowers.js script? You might have to need it, it's sort of a mock object for specialpowers in chrome:// scope.
Comment 46•10 years ago
|
||
+ loader.loadSubScript("chrome://mochikit/content/tests/SimpleTest/specialpowersAPI.js");
+ loader.loadSubScript("chrome://mochikit/content/tests/SimpleTest/SpecialPowersObserverAPI.js");
+ loader.loadSubScript("chrome://mochikit/content/tests/SimpleTest/ChromePowers.js");
+ loader.loadSubScript("chrome://mochikit/content/chrome-harness.js");
+ loader.loadSubScript("chrome://mochikit/content/browser-test.js");
+ loader.loadSubScript("chrome://mochikit/content/cc-analyzer.js");
+ loader.loadSubScript("chrome://mochikit/content/mochitest-e10s-utils.js");
12-18 00:10:05.645 I/GeckoDump( 562): ⰲ겿{"action":"log","time":1418861405646,"thread":"","js_source":"TestRunner","pid":null,"source":"mochitest","level":"INFO","message":"SimpleTest START"}ⰲ겿
12-18 00:10:05.655 I/GeckoDump( 562): TEST-UNEXPECTED-FAIL: setup.js | error parsing http://mochi.test:8888/tests.json (ReferenceError: SpecialPowers is not defined)
12-18 00:10:05.655 W/GeckoConsole( 562): [JavaScript Error: "ReferenceError: SpecialPowers is not defined" {file: "chrome://mochikit/content/tests/SimpleTest/TestRunner.js" line: 530}]
Comment 47•10 years ago
|
||
Don't you also need to load specialpowers.js?
Shouldn't you perhaps use loader.loadSubScript("chrome://specialpowers/content/specialpowersAPI.js")
instead of chrome://mochikit/content/tests/SimpleTest/?
Comment 48•10 years ago
|
||
Thanks much for the ideas, but that seems neither better nor worse:
+ loader.loadSubScript("chrome://specialpowers/content/specialpowersAPI.js");
+ loader.loadSubScript("chrome://specialpowers/content/SpecialPowersObserverAPI.js");
+ loader.loadSubScript("chrome://specialpowers/content/specialpowers.js");
+ loader.loadSubScript("chrome://specialpowers/content/ChromePowers.js");
+ loader.loadSubScript("chrome://mochikit/content/chrome-harness.js");
+ loader.loadSubScript("chrome://mochikit/content/browser-test.js");
+ loader.loadSubScript("chrome://mochikit/content/cc-analyzer.js");
+ loader.loadSubScript("chrome://mochikit/content/mochitest-e10s-utils.js");
12-18 00:33:36.404 I/GeckoDump( 563): ⰲ겿{"action":"log","time":1418862816407,"thread":"","js_source":"TestRunner","pid":null,"source":"mochitest","level":"INFO","message":"SimpleTest START"}ⰲ겿
12-18 00:33:36.414 I/GeckoDump( 563): TEST-UNEXPECTED-FAIL: setup.js | error parsing http://mochi.test:8888/tests.json (ReferenceError: SpecialPowers is not defined)
12-18 00:33:36.414 W/GeckoConsole( 563): [JavaScript Error: "ReferenceError: SpecialPowers is not defined" {file: "chrome://mochikit/content/tests/SimpleTest/TestRunner.js" line: 530}]
Assignee | ||
Comment 49•10 years ago
|
||
So, SpecialPowers is already loaded by Marionette, so you may be double-loading it, which could be bad. To make use of SpecialPowers in a Marionette call, you have to use special_powers = True, e.g., http://dxr.mozilla.org/mozilla-central/source/testing/marionette/client/marionette/tests/unit/test_import_script.py#30
Comment 50•10 years ago
|
||
(In reply to Jonathan Griffin (:jgriffin) from comment #49)
> Marionette call, you have to use special_powers = True, e.g.,
I tried that too -- no change.
Then I realized that the code complaining about SpecialPowers was being loaded from http://hg.mozilla.org/mozilla-central/annotate/5c7a6294b82a/testing/mochitest/harness.xul#l24; loading special powers from the harness made a big difference: tests are running now!
There are still some troubling errors being reported:
0 INFO SimpleTest START
1 INFO TEST-START | chrome://mochitests/content/chrome/caps/tests/mochitest/test_bug995943.xul
TEST-INFO | specialpowers.js | Failed to attach specialpowers to window exception: TypeError: SpecialPowers is not a constructor
TEST-INFO | specialpowers.js | Failed to attach specialpowers to window exception: TypeError: SpecialPowers is not a constructor
2 INFO Invoking go for window with id: 10
3 INFO TEST-PASS | chrome://mochitests/content/chrome/caps/tests/mochitest/test_bug995943.xul | Should only call go once!
4 INFO Invoking checkLoadFileURI with domain: http://example.com, shouldLoad: false
5 INFO Navigating iframe to http://example.com/tests/js/xpconnect/tests/mochitest/file_empty.html
JavaScript error: chrome://specialpowers/content/SpecialPowersObserver.js, line 103: NS_ERROR_NOT_INITIALIZED: Component returned failure code: 0xc1f30001 (NS_ERROR_NOT_INITIALIZED) [nsIMessageSender.sendAsyncMessage]
###################################### forms.js loaded
############################### browserElementPanning.js loaded
######################## BrowserElementChildPreload.js loaded
6 INFO TEST-UNEXPECTED-FAIL | chrome://mochitests/content/chrome/caps/tests/mochitest/test_bug995943.xul | Test timed out. - expected PASS
7 INFO TEST-OK | chrome://mochitests/content/chrome/caps/tests/mochitest/test_bug995943.xul | took 301138ms
TEST-INFO | specialpowers.js | Failed to attach specialpowers to window exception: TypeError: SpecialPowers is not a constructor
but this is a lot better than I have seen previously on b2g.
Assignee | ||
Comment 51•10 years ago
|
||
Note that we don't expect any tests with .xul extensions to run on B2G; the only tests that will work are .js tests.
Comment 52•10 years ago
|
||
Nice! But I'm still confused as to why mochitest-chrome even needs specialpowers in the first place? Would you like help investigating this? If so, do you mind uploading what you have so far?
Comment 53•10 years ago
|
||
(In reply to Jonathan Griffin (:jgriffin) from comment #51)
> Note that we don't expect any tests with .xul extensions to run on B2G; the
> only tests that will work are .js tests.
.js tests? Don't you mean .html tests?
(In reply to Andrew Halberstadt [:ahal] from comment #52)
> Nice! But I'm still confused as to why mochitest-chrome even needs
> specialpowers in the first place?
Because there are certain utility functions built into SpecialPowers (like pushPrefEnv) which are useful to use, even in privileged code.
Comment 54•10 years ago
|
||
special powers has a sister call chromepowers() which mimics specialpowers functionality for chrome tests. This way we can author tests and share libraries which depend on special powers.
Assignee | ||
Comment 55•10 years ago
|
||
(In reply to Bobby Holley (Busy with media, don't ask for DOM/JS/XPConnect things) from comment #53)
> (In reply to Jonathan Griffin (:jgriffin) from comment #51)
> > Note that we don't expect any tests with .xul extensions to run on B2G; the
> > only tests that will work are .js tests.
>
> .js tests? Don't you mean .html tests?
>
Yes, sorry...I mean .html tests are the only ones we expect to be able to run on B2G.
Comment 56•10 years ago
|
||
This is how I'm running mochitest-chrome on b2g. It basically sets options.chrome and then does everything the same as "mochitest-remote".
Comment 57•10 years ago
|
||
As noted in the comment, there's a hack here; I need to re-work getChromeTestDir to make this cleaner.
Comment 58•10 years ago
|
||
Comment 59•10 years ago
|
||
These are the changes I used to kick off the harness load and get the results I showed in Comment 50. Obviously this needs work!
When I updated my repo today and re-tested these changes, they stopped working -- tests hang and I don't see any errors. I don't know where this is going wrong.
Comment 60•10 years ago
|
||
(In reply to Andrew Halberstadt [:ahal] from comment #52)
> Nice! But I'm still confused as to why mochitest-chrome even needs
> specialpowers in the first place? Would you like help investigating this? If
> so, do you mind uploading what you have so far?
I agree with the sentiments of Bobby and Joel: We're sharing code between plain mochitests and chrome -- specialpowers dependencies are a natural consequence. I'm not very concerned about that.
I *am* concerned that I need to load specialpowers when I don't on other platforms -- I think that's a sign that tests are accidentally running in a different environment, and I think that's a consequence of the terrible hacks in my 4th patch. Suggestions for how to avoid these hacks and accomplish the harness load in a reasonable way are more than welcome!
Comment 61•10 years ago
|
||
(In reply to Geoff Brown [:gbrown] from comment #59)
> When I updated my repo today and re-tested these changes, they stopped
> working -- tests hang and I don't see any errors. I don't know where this is
> going wrong.
Actually, I think there is just a long delay, rather than an infinite hang.
$ ./mach mochitest-chrome-remote layout/base/tests/chrome/test_bug370436.html
...
0 INFO SimpleTest START
1 INFO TEST-START | layout/base/tests/chrome/test_bug370436.html
TEST-INFO | specialpowers.js | Failed to attach specialpowers to window exception: TypeError: SpecialPowers is not a constructor
TEST-INFO | specialpowers.js | Failed to attach specialpowers to window exception: TypeError: SpecialPowers is not a constructor
2 INFO must wait for load
3 INFO TEST-PASS | layout/base/tests/chrome/test_bug370436.html | Shouldn't have SOWs in chrome
4 INFO TEST-PASS | layout/base/tests/chrome/test_bug370436.html | Shouldn't have SOWs in chrome
5 INFO TEST-PASS | layout/base/tests/chrome/test_bug370436.html | Shouldn't have SOWs in chrome
6 INFO TEST-PASS | layout/base/tests/chrome/test_bug370436.html | Shouldn't have SOWs in chrome
7 INFO TEST-PASS | layout/base/tests/chrome/test_bug370436.html | Shouldn't have SOWs in chrome
8 INFO TEST-PASS | layout/base/tests/chrome/test_bug370436.html | Shouldn't have SOWs in chrome
9 INFO TEST-PASS | layout/base/tests/chrome/test_bug370436.html | Shouldn't have SOWs in chrome
10 INFO TEST-PASS | layout/base/tests/chrome/test_bug370436.html | Word 1 selected correctly
11 INFO TEST-PASS | layout/base/tests/chrome/test_bug370436.html | Word 2 selected correctly
12 INFO TEST-PASS | layout/base/tests/chrome/test_bug370436.html | Word 3 selected correctly
13 INFO TEST-PASS | layout/base/tests/chrome/test_bug370436.html | Word 4 selected correctly
14 INFO TEST-PASS | layout/base/tests/chrome/test_bug370436.html | Word 5 selected correctly
15 INFO TEST-PASS | layout/base/tests/chrome/test_bug370436.html | Word 6 selected correctly
16 INFO TEST-PASS | layout/base/tests/chrome/test_bug370436.html | Word 7 selected correctly
17 INFO MEMORY STAT vsize after test: 150593536
18 INFO MEMORY STAT vsizeMaxContiguous not supported in this build configuration.
19 INFO MEMORY STAT residentFast after test: 55218176
20 INFO MEMORY STAT heapAllocated after test: 25556628
21 INFO TEST-OK | layout/base/tests/chrome/test_bug370436.html | took 1722ms
22 INFO TEST-START | Shutdown
23 INFO Passed: 14
24 INFO Failed: 0
25 INFO Todo: 0
26 INFO Slowest: 1721ms - chrome://mochitests/content/chrome/layout/base/tests/chrome/test_bug370436.html
27 INFO SimpleTest FINISHED
28 INFO TEST-INFO | Ran 1 Loops
29 INFO SimpleTest FINISHED
-*- NetworkService: NetworkService shutdown
[Parent 352] WARNING: waitpid failed pid:359 errno:10: file /home/gbrown/B2G2/gecko/ipc/chromium/src/base/process_util_posix.cc, line 267
JavaScript error: resource://gre/modules/IndexedDBHelper.jsm, line 121: UnknownError: The operation failed for reasons unrelated to the database itself and not covered by any other error code.
TEST-INFO | leakcheck | default process: leak threshold set at 5180 bytes
TEST-INFO | leakcheck | plugin process: leak threshold set at 0 bytes
TEST-INFO | leakcheck | tab process: leak threshold set at 2000000 bytes
TEST-INFO | leakcheck | geckomediaplugin process: leak threshold set at 20000 bytes
TEST-INFO | leakcheck | default process: ignoring missing output line for total leaks
Stopping web server
Stopping web socket server
Stopping ssltunnel
runtestsb2g.py | Running tests: end.
SUITE-END | took 164s
Comment 62•10 years ago
|
||
Adds the mochitest-chrome-remote mach command and avoids a check that prevents specifying a test path for chrome.
Attachment #8538800 -
Attachment is obsolete: true
Comment 63•10 years ago
|
||
Comment on attachment 8538866 [details] [diff] [review]
wip - add "mochitest-chrome-remote" mach command
Review of attachment 8538866 [details] [diff] [review]:
-----------------------------------------------------------------
::: testing/mochitest/mach_commands.py
@@ +843,5 @@
> xre_path=self.xre_path, test_paths=test_paths, **kwargs)
>
> + @Command('mochitest-chrome-remote', category='testing',
> + description='Run a remote mochitest-chrome.',
> + conditions=[conditions.is_b2g, is_emulator])
This is fine for now since mochitest-plain does the same thing, but I regret starting this pattern and filed bug 1046992 to move away from it. You have enough on your plate so no need to worry about it now, we can fix it as part of the other bug.
Comment 64•10 years ago
|
||
(In reply to Geoff Brown [:gbrown] from comment #61)
> (In reply to Geoff Brown [:gbrown] from comment #59)
> > When I updated my repo today and re-tested these changes, they stopped
> > working -- tests hang and I don't see any errors. I don't know where this is
> > going wrong.
>
> Actually, I think there is just a long delay, rather than an infinite hang.
Actually, I was just anxious and not reading logs well. It takes a while to start the emulator and push files, but the test execution is fine.
Comment 65•10 years ago
|
||
Aha!
I was getting the nsIWebNavigation reference from the wrong window, when loading the harness. If instead I use:
let homescreen = document.getElementById('systemapp');
var webNav = homescreen.contentWindow.QueryInterface(Ci.nsIInterfaceRequestor)
.getInterface(Ci.nsIWebNavigation);
webNav.loadURI(url, null, null, null, null);
then things work better: 1. The harness and test are visible in the emulator; 2. The specialpowers hacks are no longer necessary. Yeah!
Comment 66•10 years ago
|
||
As noted earlier, this is probably not the right long-term solution, but bug 1046992 should clear that up.
For now, I just want *some* simple way to start mochitest-chrome tests on B2G.
My default test command line is:
./mach mochitest-chrome-remote layout/base/tests/chrome/test_bug370436.html
Attachment #8538866 -
Attachment is obsolete: true
Attachment #8540760 -
Flags: review?(ahalberstadt)
Comment 67•10 years ago
|
||
This pushes the chrome test files to the remote device.
An unfortunate complication arises because the location of the chrome test files is written to the profile. The profile is passed to Marionette, so the profile must be built before using the device manager initialized in the Marionette runner. But the device manager should be queried to determine the remote test root used for the location of the chrome test files. In this patch, I provide a dummy location for the chrome test files until self.app_ctx is initialized, and update the profile with the correct location after Marionette is initialized.
Attachment #8538807 -
Attachment is obsolete: true
Attachment #8540765 -
Flags: review?(ahalberstadt)
Comment 68•10 years ago
|
||
Simplified.
Attachment #8538811 -
Attachment is obsolete: true
Attachment #8538813 -
Attachment is obsolete: true
Comment 69•10 years ago
|
||
Comment on attachment 8540760 [details] [diff] [review]
add "mochitest-chrome-remote" mach command
Review of attachment 8540760 [details] [diff] [review]:
-----------------------------------------------------------------
Cool, lgtm!
Attachment #8540760 -
Flags: review?(ahalberstadt) → review+
Comment 70•10 years ago
|
||
I've tried to keep this simple and avoid impacting plain mochitests on b2g or mochitest-chrome on other platforms: Pass chrome option to b2g_start_script; if b2g_start_script called with chrome option, load browser-test.js and call b2gStart() to load harness and start chrome tests.
Attachment #8540801 -
Attachment is obsolete: true
Attachment #8540866 -
Flags: review?(ahalberstadt)
Comment 71•10 years ago
|
||
Comment on attachment 8540765 [details] [diff] [review]
push chrome test files to device
Review of attachment 8540765 [details] [diff] [review]:
-----------------------------------------------------------------
Looks good. I'm assuming this isn't the whole thing needed to get it working.. There should be some changes to b2g_start_script.js or a new script that is similar, right?
::: testing/mochitest/runtestsb2g.py
@@ +190,5 @@
> Services.io.offline = false;
> """)
>
> + if options.chrome:
> + self.app_ctx.dm.removeDir(self.remote_chrome_test_dir)
Shouldn't self.remote_chrome_test_dir get defined and set to None in __init__ or in the class namespace? Otherwise this will raise if there have been no calls to getChromeTestDir() yet.
@@ +248,5 @@
> + # writing the dummy.
> + if hasattr(self, 'app_ctx'):
> + self.remote_chrome_test_dir = posixpath.join(self.app_ctx.remote_test_root, 'chrome');
> + return self.remote_chrome_test_dir
> + return 'dummy-chrome-test-dir'
I haven't looked into this problem yet, so maybe this is the best way forward, but modifications to mozrunner and/or marionette are totally on the table if there's a cleaner fix involving changes to those.
Attachment #8540765 -
Flags: review?(ahalberstadt) → review+
Comment 72•10 years ago
|
||
Comment on attachment 8540866 [details] [diff] [review]
instead of loading via browser-test-overlay.xul, simply load harness on b2g
Review of attachment 8540866 [details] [diff] [review]:
-----------------------------------------------------------------
::: testing/mochitest/b2g_start_script.js
@@ +89,5 @@
>
> +if (chrome) {
> + let loader = Cc["@mozilla.org/moz/jssubscript-loader;1"].getService(Ci.mozIJSSubScriptLoader);
> + loader.loadSubScript("chrome://mochikit/content/browser-test.js");
> + b2gStart();
Is b2gStart what actually kicks off the tests? If so, should we avoid setting container.src = mochitestUrl below?
Attachment #8540866 -
Flags: review?(ahalberstadt) → review+
Comment 73•10 years ago
|
||
(In reply to Andrew Halberstadt [:ahal] from comment #71)
> Shouldn't self.remote_chrome_test_dir get defined and set to None in
> __init__ or in the class namespace? Otherwise this will raise if there have
> been no calls to getChromeTestDir() yet.
Good idea.
> @@ +248,5 @@
> > + # writing the dummy.
> > + if hasattr(self, 'app_ctx'):
> > + self.remote_chrome_test_dir = posixpath.join(self.app_ctx.remote_test_root, 'chrome');
> > + return self.remote_chrome_test_dir
> > + return 'dummy-chrome-test-dir'
>
> I haven't looked into this problem yet, so maybe this is the best way
> forward, but modifications to mozrunner and/or marionette are totally on the
> table if there's a cleaner fix involving changes to those.
I thought about trying to call down to get_dm in b2g.py, but that's really buried under several layers. I also considered pulling the devicemanager access out of the runner and into some separate class, but I didn't think that effort was warranted when I could work around it.
Comment 74•10 years ago
|
||
(In reply to Andrew Halberstadt [:ahal] from comment #72)
> Is b2gStart what actually kicks off the tests? If so, should we avoid
> setting container.src = mochitestUrl below?
Yes, b2gStart kicks off the tests. Setting container.src has not caused me any trouble, but I agree it seems unsafe...I'll avoid it in the final patch.
Comment 75•10 years ago
|
||
Try runs suggest these cause no harm to existing jobs:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=822914cc9704
https://treeherder.mozilla.org/#/jobs?repo=try&revision=5bb3e6556652
Comment 76•10 years ago
|
||
Updated•10 years ago
|
Keywords: leave-open
Comment 77•10 years ago
|
||
Comment 78•10 years ago
|
||
Thank you for doing this!
Does this mean that any .html mochitest-chrome test will now run by default on b2g? If so, that's worth an announcement on dev-platform.
Flags: needinfo?(gbrown)
Comment 79•10 years ago
|
||
Once a job is created and scheduled on the CI system. These changes allow for mochitest-chrome tests to be run on b2g. I know this was developed/tested on an emulator, so all b2g builds/devices we test on could take an extra fix or two.
Comment 80•10 years ago
|
||
(In reply to Bobby Holley (Busy with media, don't ask for DOM/JS/XPConnect things) from comment #78)
> Does this mean that any .html mochitest-chrome test will now run by default on b2g?
With these patches, my claim is that we can run mochitest-chrome tests on b2g emulator via
mach mochitest-chrome-remote
I've spot-tested a few .html tests with good results, but I have not tested extensively. I would expect some (many?) tests to fail -- it seems like every test suite needs some TLC before it will run reliably on a new platform.
I think :jgriffin has the right idea with bug 1116187: Let's see this running on try next.
Flags: needinfo?(gbrown)
Updated•10 years ago
|
Assignee | ||
Comment 81•10 years ago
|
||
Assigning to myself for the final scheduling bits.
Assignee: gbrown → jgriffin
Assignee | ||
Comment 82•10 years ago
|
||
Status: REOPENED → RESOLVED
Closed: 12 years ago → 10 years ago
Resolution: --- → FIXED
Assignee | ||
Updated•10 years ago
|
Keywords: leave-open
Comment 83•9 years ago
|
||
On bug 1171917 I'm adding a new chrome test for B2G. If I push to try and use the following try syntax 'try: -b o -p emulator -u mochitest-chrome -t none' the results show up on try. On the other hand when using 'try: -b do -p all -u all -t none' before landing I do not see the chrome tests results. Are these b2g chrome test results checked when landing (I mean checking in)? Thanks!
Flags: needinfo?(jgriffin)
Comment 84•9 years ago
|
||
Sorry, I see them. I didn't notice the results show up when landing as well. Thanks!
Flags: needinfo?(jgriffin)
Updated•7 years ago
|
Component: Mochitest Chrome → Mochitest
You need to log in
before you can comment on or make changes to this bug.
Description
•