Closed
Bug 958123
Opened 11 years ago
Closed 11 years ago
[marketplace-tests-gaia] Override the launch method in GaiaApps to ignore pesky Javascript errors
Categories
(Firefox OS Graveyard :: Gaia::UI Tests, defect, P3)
Tracking
(Not tracked)
RESOLVED
WONTFIX
People
(Reporter: bsilverberg, Assigned: bsilverberg)
References
Details
(Whiteboard: [marketplace-tests-gaia])
Attachments
(1 file)
There have been a lot of errors on b2g.hamachi.mozilla-aurora.marketplace, most of which have been a JavascriptException that is thrown during the launch of the Marketplace app, which Marionette then catches and causes the launch to fail. This is a non-critical JS exception, but it causes the test to fail.
A sample stack trace is:
test_marketplace_feedback_user (test_marketplace_feedback_login.TestMarketplaceFeedback) ... ERROR
======================================================================
ERROR: None
----------------------------------------------------------------------
Traceback (most recent call last):
File "/var/jenkins/workspace/b2g.hamachi.mozilla-aurora.marketplace/.env/local/lib/python2.7/site-packages/marionette_client-0.7.1-py2.7.egg/marionette/marionette_test.py", line 143, in run
testMethod()
File "/var/jenkins/workspace/b2g.hamachi.mozilla-aurora.marketplace/marketplacetests/tests/test_marketplace_feedback_login.py", line 24, in test_marketplace_feedback_user
self.marketplace.launch()
File "/var/jenkins/workspace/b2g.hamachi.mozilla-aurora.marketplace/marketplacetests/marketplace/app.py", line 48, in launch
Base.launch(self, launch_timeout=120000)
File "/var/jenkins/workspace/b2g.hamachi.mozilla-aurora.marketplace/.env/local/lib/python2.7/site-packages/gaiatest-0.21.2-py2.7.egg/gaiatest/apps/base.py", line 26, in launch
self.app = self.apps.launch(self.name, launch_timeout=launch_timeout)
File "/var/jenkins/workspace/b2g.hamachi.mozilla-aurora.marketplace/.env/local/lib/python2.7/site-packages/gaiatest-0.21.2-py2.7.egg/gaiatest/gaia_test.py", line 75, in launch
result = self.marionette.execute_async_script("GaiaApps.launchWithName('%s')" % name, script_timeout=launch_timeout)
File "/var/jenkins/workspace/b2g.hamachi.mozilla-aurora.marketplace/.env/local/lib/python2.7/site-packages/marionette_client-0.7.1-py2.7.egg/marionette/marionette.py", line 1073, in execute_async_script
filename=os.path.basename(frame[0]))
File "/var/jenkins/workspace/b2g.hamachi.mozilla-aurora.marketplace/.env/local/lib/python2.7/site-packages/marionette_client-0.7.1-py2.7.egg/marionette/marionette.py", line 577, in _send_message
self._handle_error(response)
File "/var/jenkins/workspace/b2g.hamachi.mozilla-aurora.marketplace/.env/local/lib/python2.7/site-packages/marionette_client-0.7.1-py2.7.egg/marionette/marionette.py", line 612, in _handle_error
raise JavascriptException(message=message, status=status, stacktrace=stacktrace)
JavascriptException: TypeError: widgetFrame is null
stacktrace:
@app://system.gaiamobile.org/js/cost_control.js, line 70
TEST-UNEXPECTED-FAIL | test_marketplace_feedback_login.py test_marketplace_feedback_login.TestMarketplaceFeedback.test_marketplace_feedback_user |
----------------------------------------------------------------------
Ran 1 test in 74.436s
FAILED (errors=1)
We saw something similar in the gaia-ui-tests job awhile back and had to wait for a developer to address it.
Because this repo is small and focussed on testing Marketplace functionality, as opposed to general gaia reliability, it was suggested that maybe we should just ignore these errors. I believe that Bebe proposed a similar approach at the time and it wasn't favoured, but I discussed this with Zac on IRC and we thought it might be appropriate for this repo.
Assignee | ||
Comment 1•11 years ago
|
||
Attachment #8357840 -
Flags: review?(zcampbell)
Attachment #8357840 -
Flags: review?(viorela.ioia)
Attachment #8357840 -
Flags: review?(trifandreialin)
Attachment #8357840 -
Flags: review?(florin.strugariu)
Attachment #8357840 -
Flags: review?(andrei.hutusoru)
Comment 2•11 years ago
|
||
Comment on attachment 8357840 [details]
Pull request
See comments in pull request.
Attachment #8357840 -
Flags: review-
Comment 3•11 years ago
|
||
Comment on attachment 8357840 [details]
Pull request
Bob, sorry I don't have time to run/review this thoroughly but I'm switching my r? to a f+ for the principle of the pull and I'll let you guys hash out the details of code syntax.
Attachment #8357840 -
Flags: review?(zcampbell) → feedback+
Comment 4•11 years ago
|
||
Comment on attachment 8357840 [details]
Pull request
+1 to some of Dave's comments
Attachment #8357840 -
Flags: review?(florin.strugariu) → review-
Assignee | ||
Comment 5•11 years ago
|
||
Comment on attachment 8357840 [details]
Pull request
Newer version of the PR.
Attachment #8357840 -
Flags: review?(zcampbell)
Attachment #8357840 -
Flags: review?(florin.strugariu)
Attachment #8357840 -
Flags: review?(dave.hunt)
Attachment #8357840 -
Flags: review-
Comment 6•11 years ago
|
||
Comment on attachment 8357840 [details]
Pull request
f+ only, no time to review sorry.
Attachment #8357840 -
Flags: review?(zcampbell)
Comment 7•11 years ago
|
||
Comment on attachment 8357840 [details]
Pull request
I've not run the tests but the patch looks okay to me. I still think we're fixing (hacking) this in the wrong place though as the exception mentioned is not part of Marketplace.
Attachment #8357840 -
Flags: review?(dave.hunt) → review+
Comment 8•11 years ago
|
||
If the failure isn't related to marketplace I don't understand why other aurora tests aren't failing.
Comment 9•11 years ago
|
||
(In reply to Zac C (:zac) from comment #8)
> If the failure isn't related to marketplace I don't understand why other
> aurora tests aren't failing.
True, but what has cost_control.js got to do with Marketplace? I think we need more investigation here.
Assignee | ||
Comment 10•11 years ago
|
||
After all of this discussion and debate, the job seems to have stabilized. We haven't seen a JavascriptException in a couple of days, so let's hold off on this change for now. If things are still stable by the end of the week we should consider abandoning this.
Comment 11•11 years ago
|
||
Comment on attachment 8357840 [details]
Pull request
LGTM
Attachment #8357840 -
Flags: review?(florin.strugariu) → review+
Comment 12•11 years ago
|
||
Merging in
Comment 13•11 years ago
|
||
Merged in https://github.com/mozilla/marketplace-tests-gaia/commit/c16f83ff799e4cfdf7243a7c699c42ccaf7fe362
Status: ASSIGNED → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
Updated•11 years ago
|
Attachment #8357840 -
Flags: review?(viorela.ioia)
Attachment #8357840 -
Flags: review?(trifandreialin)
Attachment #8357840 -
Flags: review?(andrei.hutusoru)
Comment 14•11 years ago
|
||
This shouldn't have been merged yet based on comment 10. I'll leave it to Bob to decide if we want to back this out though.
Comment 15•11 years ago
|
||
Sorry about that guys I did not seen comment 10. I backed out the commit and we will wait until next wee as Bob suggested.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Assignee | ||
Comment 17•11 years ago
|
||
Hmm, we did see one of these Javascript Exceptions [1] the day after those last comments were posted. So I guess the problem has not been eradicated, but it is definitely less frequent than we were seeing early last week. In the past 22 builds we've only seen one test fail with this Javascript exception, so it sounds like maybe we're better off not merging this hack just yet.
What do you think, Bebe and davehunt?
[1] http://qa-selenium.mv.mozilla.com:8080/view/B2G%20Hamachi/job/b2g.hamachi.mozilla-aurora.marketplace/25/HTML_Report/?
Flags: needinfo?(dave.hunt)
Comment 18•11 years ago
|
||
My preference is not to implement the workaround but to fix the issue. Also, if we implement the workaround I still think it should be in gaia, because as far as I can tell the issue is not specific to Marketplace.
Flags: needinfo?(dave.hunt)
Comment 19•11 years ago
|
||
In the last 30 builds (1 week) we saw 11 failed builds from these only 2 builds failed with a JavascriptException()
I would not want to implement a hack for this.
Let's log a bug to the Marketplace project so they could investigate the issue.
Please reopen this if we see the fail in a more often case
Status: REOPENED → RESOLVED
Closed: 11 years ago → 11 years ago
Flags: needinfo?(florin.strugariu)
Resolution: --- → WONTFIX
You need to log in
before you can comment on or make changes to this bug.
Description
•