While running only the following benchmark [1], while collecting results for are we fast yet, I git the following error message: ROUND 0 ------- running webserver on http://10.250.1.241:37478/ TEST-START test_bench_sunspider.py test_sunspider (test_bench_sunspider.TestBenchSunspider) ... ERROR ERROR ====================================================================== ERROR: test_sunspider (test_bench_sunspider.TestBenchSunspider) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/awsa/unagi/B2G/gecko/testing/marionette/client/marionette/marionette_test.py", line 132, in run testMethod() File "/home/awsa/unagi/B2G/perso/gaia-ui-tests/gaiatest/tests/browser/benchmarks/test_bench_sunspider.py", line 30, in test_sunspider browser.go_to_url(self._start_page) File "/home/awsa/unagi/B2G/perso/gaia-ui-tests/gaiatest/apps/browser/app.py", line 55, in go_to_url self.keyboard.send(url) File "/home/awsa/unagi/B2G/perso/gaia-ui-tests/gaiatest/apps/keyboard/app.py", line 173, in send self._switch_to_correct_layout(val) File "/home/awsa/unagi/B2G/perso/gaia-ui-tests/gaiatest/apps/keyboard/app.py", line 89, in _switch_to_correct_layout self._tap(self._alpha_key) File "/home/awsa/unagi/B2G/perso/gaia-ui-tests/gaiatest/apps/keyboard/app.py", line 117, in _tap Actions(self.marionette).press(key).wait(0.1).release().perform() File "/home/awsa/unagi/B2G/gecko/testing/marionette/client/marionette/marionette.py", line 362, in perform self.current_id = self.marionette._send_message('actionChain', 'value', chain=self.action_chain, nextId=self.current_id) File "/home/awsa/unagi/B2G/gecko/testing/marionette/client/marionette/marionette.py", line 570, in _send_message self._handle_error(response) File "/home/awsa/unagi/B2G/gecko/testing/marionette/client/marionette/marionette.py", line 631, in _handle_error raise MarionetteException(message=message, status=status, stacktrace=stacktrace) TEST-UNEXPECTED-FAIL | test_bench_sunspider.py TestBenchSunspider.test_sunspider | MarionetteException: Component returned failure code: 0x80004003 (NS_ERROR_INVALID_POINTER) [nsIDOMWindowUtils.sendTouchEvent] ====================================================================== ERROR: test_sunspider (test_bench_sunspider.TestBenchSunspider) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/awsa/unagi/B2G/gecko/testing/marionette/client/marionette/marionette_test.py", line 162, in run self.tearDown() File "/home/awsa/unagi/B2G/perso/gaia-ui-tests/gaiatest/gaia_test.py", line 675, in tearDown MarionetteTestCase.tearDown(self) File "/home/awsa/unagi/B2G/gecko/testing/marionette/client/marionette/marionette_test.py", line 307, in tearDown self.marionette.set_context("content") File "/home/awsa/unagi/B2G/gecko/testing/marionette/client/marionette/marionette.py", line 796, in set_context return self._send_message('setContext', 'ok', value=context) File "/home/awsa/unagi/B2G/gecko/testing/marionette/client/marionette/marionette.py", line 546, in _send_message raise MarionetteException(message="Please start a session") TEST-UNEXPECTED-FAIL | test_bench_sunspider.py TestBenchSunspider.test_sunspider | MarionetteException: Please start a session ---------------------------------------------------------------------- Ran 1 test in 266.664s FAILED (errors=2) SUMMARY ------- passed: 0 failed: 2 todo: 0 FAILED TESTS ------- [1] https://github.com/nbp/gaia-ui-tests/blob/bench/gaiatest/tests/browser/benchmarks/test_bench_sunspider.py

I'm running this on today's mozilla-central build on an inari and it is passing. This is a non-inbound build, so this verifies the test is working correctly with a recent version of mozilla-central. Since nothing has been changed in marionette today, then it is most likely that the build on mozilla-inbound that you're running against may be broken.

Just saw this on the nightly m-c/unagi, test against the browser. It's not 100% repeatable. Have they tweaked the events in browser app lately?

(In reply to Malini Das [:mdas] from comment #1) > Since nothing has been changed in marionette today, then it is most likely > that the build on mozilla-inbound that you're running against may be broken. This might not be from today, since I accidentally switch-off the update mechanism on Friday evening. So the regression might be between Friday and today. I also had to rebase the previous benchmark on top of the latest gaia-ui-test since some locator did not seems to work anymore. (In reply to Zac C (:zac) from comment #2) > It's not 100% repeatable. It is 100% reproducible on hydra (AWFY build mahcine for the Unagi). Are the failing device always the same?

I first saw the error on Gecko ab0a4b4b9b1fd684cc38fe144782b5adb73a9985, and Gaia abe3575. This might not be the first broken pair. As a new regression landed, I will generate Unagi images. If you need any, I can make them available within the Mozilla network.

This couldn't be reproduced in recent builds, must have been resolved. Please reopen if it happens again with specific build information (build device, date of build, commit hashes of gecko/gaia and maybe even a pvtbuilds url to the build if applicable)

I did a bisect of this bug while investigating for Bug 915167, It seems that the bug which cause this error is: Bug 913424 - IonMonkey: Handle JSOP_THIS primitive this case. r=bhackett Changeset d660739f7498 This seems might be related to Sunspider crashes.