This is currently closing b2g26. We originally thought this was related to some keyboard uplifts, but we can still reproduce hitting this failure even post-backout. Need someone who better understands these tests to take a look. https://tbpl-dev.allizom.org/php/getParsedLog.php?id=29965593&tree=Mozilla-B2g26-v1.2 b2g_ubuntu64_vm mozilla-b2g26_v1_2 opt test gaia-ui-test on 2013-11-01 05:14:42 PDT for push 163f34a949e9 slave: tst-linux64-ec2-309 05:24:06 INFO - Traceback (most recent call last): 05:24:06 INFO - File "/builds/slave/test/gaia/tests/python/gaia-ui-tests/gaiatest/cli.py", line 4, in <module> 05:24:06 INFO - main() 05:24:06 INFO - File "/builds/slave/test/gaia/tests/python/gaia-ui-tests/gaiatest/runtests.py", line 352, in main 05:24:06 INFO - cli(runner_class=GaiaTestRunner, parser_class=GaiaTestOptions) 05:24:06 INFO - File "/builds/slave/test/build/venv/local/lib/python2.7/site-packages/marionette/runtests.py", line 812, in cli 05:24:06 INFO - runner = startTestRunner(runner_class, options, tests) 05:24:06 INFO - File "/builds/slave/test/build/venv/local/lib/python2.7/site-packages/marionette/runtests.py", line 804, in startTestRunner 05:24:06 INFO - runner.run_tests(tests) 05:24:06 INFO - File "/builds/slave/test/gaia/tests/python/gaia-ui-tests/gaiatest/runtests.py", line 216, in run_tests 05:24:06 INFO - MarionetteTestRunner.run_tests(self, tests) 05:24:06 INFO - File "/builds/slave/test/build/venv/local/lib/python2.7/site-packages/marionette/runtests.py", line 430, in run_tests 05:24:06 INFO - self.run_test(test) 05:24:06 INFO - File "/builds/slave/test/build/venv/local/lib/python2.7/site-packages/marionette/runtests.py", line 523, in run_test 05:24:06 INFO - self.run_test(i["path"]) 05:24:06 INFO - File "/builds/slave/test/build/venv/local/lib/python2.7/site-packages/marionette/runtests.py", line 544, in run_test 05:24:06 INFO - results = runner.run(suite) 05:24:06 INFO - File "/builds/slave/test/build/venv/local/lib/python2.7/site-packages/marionette/runtests.py", line 155, in run 05:24:06 INFO - test(result) 05:24:06 INFO - File "/usr/lib/python2.7/unittest/suite.py", line 70, in __call__ 05:24:06 INFO - return self.run(*args, **kwds) 05:24:06 INFO - File "/usr/lib/python2.7/unittest/suite.py", line 108, in run 05:24:06 INFO - test(result) 05:24:06 INFO - File "/usr/lib/python2.7/unittest/case.py", line 391, in __call__ 05:24:06 INFO - return self.run(*args, **kwds) 05:24:06 INFO - File "/builds/slave/test/build/venv/local/lib/python2.7/site-packages/marionette/marionette_test.py", line 158, in run 05:24:06 INFO - result.addError(self, sys.exc_info()) 05:24:06 INFO - File "/builds/slave/test/gaia/tests/python/gaia-ui-tests/gaiatest/runtests.py", line 56, in addError 05:24:06 INFO - debug=self.gather_debug()) 05:24:06 INFO - File "/builds/slave/test/gaia/tests/python/gaia-ui-tests/gaiatest/runtests.py", line 50, in add_result 05:24:06 INFO - self.append(t) 05:24:06 INFO - AttributeError: 'GaiaTestResult' object has no attribute 'append' 05:24:06 INFO - 1383308646567 Marionette INFO sendToClient: {"from":"0","value":{"accessibility.invert":false,"accessibility.screenreader":false,"alarm.enabled":true,"app.launch_path.blacklist":[],"app.reportCrashes":"ask","app.update.interval":86400,"audio.volume.alarm":0,"audio.volume.bt_sco":0,"audio.volume.cemaxvol":11,"audio.volume.content":0,"audio.volume.dtmf":15,"audio.volume.master":0,"audio.volume.notification":0,"audio.volume.telephony":0,"audio.volume.tts":15,"bluetooth.debugging.enabled":false,"bluetooth.enabled":false,"bluetooth.suspended":false,"camera.recording.preferredSizes":[],"camera.recordingsound.enabled":false,"camera.shutter.enabled":true,"clear.remote-windows.data":false,"debug.console.enabled":true,"debug.fps.enabled":false,"debug.gaia.enabled":false,"debug.grid.enabled":false,"debug.log-animations.enabled":false,"debug.oop.disabled":false,"debug.paint-flashing.enabled":false,"debug.peformancedata.shared":false,"debug.ttl.enabled":false,"device.storage.writable.name":"sdcard","deviceinfo.firmware_revision":null,"deviceinfo.hardware":null,"deviceinfo.mac":"","deviceinfo.os":"1.2.0.0-prerelease","deviceinfo.platform_build_id":"20131031133244","deviceinfo.platform_version":"26.0","deviceinfo.product_model":null,"deviceinfo.software":"Boot2Gecko 1.2.0.0-prerelease","deviceinfo.update_channel":"default","devtools.debugger.remote-enabled":false,"dialer.ringtone":{"type":"audio/ogg","size":31194},"dom.mms.operatorSizeLimitation":307200,"dom.mozContacts.debugging.enabled":false,"feedback.url":"https://input.allizom.org/api/v1/feedback/","gaia.system.checkForUpdates":false,"geolocation.enabled":true,"geolocation.suspended":false,"homegesture.enabled":true,"homescreen.manifestURL":"app://homescreen.gaiamobile.org/manifest.webapp","icc.applications":"{}","icc.data":null,"icc.displayTextTimeout":40000,"icc.goBackTimeout":1000,"icc.inputTextTimeout":40000,"icc.selectTimeout":150000,"keyboard.autocorrect":true,"keyboard.clicksound":false,"keyboard.current":"en","keyboard.default-layouts":{"app://keyboard.gaiamobile.org":{"en":true,"number":true}},"keyboard.enabled-layouts":{"app://keyboard.gaiamobile.org":{"en":true,"number":true}},"keyboard.ftu.enabled":false,"keyboard.layouts.arabic":false,"keyboard.layouts.catalan":false,"keyboard.layouts.czech":false,"keyboard.layouts.dvorak":false,"keyboard.layouts.english":true,"keyboard.layouts.french":false,"keyboard.layouts.german":false,"keyboard.layouts.greek":false,"keyboard.layouts.hebrew":false,"keyboard.layouts.hungarian":false,"keyboard.layouts.japanese":false,"keyboard.layouts.norwegian":false,"keyboard.layouts.pinyin":false,"keyboard.layouts.polish":false,"keyboard.layouts.portuguese":false,"keyboard.layouts.romanian":false,"keyboard.layouts.russian":false,"keyboard.layouts.serbian":false,"keyboard.layouts.slovak":false,"keyboard.layouts.spanish":false,"keyboard.layouts.turkish":false,"keyboard.layouts.zhuyin":false,"keyboard.vibration":false,"keyboard.wordsuggestion":true,"language.current":"en-US","layers.draw-borders":false,"lockscreen.enabled":true,"lockscreen.locked":false,"lockscreen.notifications-preview.enabled":true,"lockscreen.passcode-lock.code":"1111","lockscreen.passcode-lock.enabled":false,"lockscreen.passcode-lock.timeout":0,"lockscreen.unlock-sound.enabled":false,"mail.sent-sound.enabled":true,"message.sent-sound.enabled":true,"notification.ringtone":{"type":"audio/ogg","size":9790},"operatorvariant.mcc":"000","operatorvariant.mnc":"00","phone.dtmf.type":"long","phone.ring.keypad":true,"powersave.enabled":false,"powersave.threshold":-1,"privacy.donottrackheader.enabled":false,"privacy.donottrackheader.value":"-1","ril.callerId":"CLIR_DEFAULT","ril.callwaiting.enabled":null,"ril.cellbroadcast.disabled":false,"ril.cellbroadcast.searchlist":"","ril.cf.enabled":false,"ril.data.apn":"","ril.data.apnSettings":"","ril.data.authtype":"notDefined","ril.data.carrier":"","ril.data.enabled":false,"ril.data.httpProxyHost":"","ril.data.httpProxyPort":0,"ril.data.mmsc":"","ril.data.mmsport":0,"ril.data.mmsproxy":"","ril.data.passwd":"","ril.data.roaming_enabled":false,"ril.data.suspended":false,"ril.data.user":"","ril.iccInfo.mbdn":"","ril.mms.apn":"","ril.mms.authtype":"notDefined","ril.mms.carrier":"","ril.mms.httpProxyHost":"","ril.mms.httpProxyPort":"","ril.mms.mmsc":"","ril.mms.mmsport":"","ril.mms.mmsproxy":"","ril.mms.passwd":"","ril.mms.retrieval_mode":"automatic-home","ril.mms.user":"","ril.radio.disabled":false,"ril.radio.preferredNetworkType":"","ril.sms.requestStatusReport.enabled":false,"ril.sms.strict7BitEncoding.enabled":false,"ril.supl.apn":"","ril.supl.authtype":"notDefined","ril.supl.carrier":"","ril.supl.httpProxyHost":"","ril.supl.httpProxyPort":"","ril.supl.passwd":"","ril.supl.user":"","screen.automatic-brightness":true,"screen.brightness":1,"screen.timeout":0,"software-button.enabled":false,"support.callsupport1.href":"","support.callsupport1.title":"","support.callsupport2.href":"","support.callsupport2.title":"","support.onlinesupport.href":"","support.onlinesupport.title":"","telephony.speaker.enabled":false,"tethering.usb.connectedClients":0,"tethering.usb.dhcpserver.endip":"192.168.0.30","tethering.usb.dhcpserver.startip":"192.168.0.10","tethering.usb.enabled":false,"tethering.usb.ip":"192.168.0.1","tethering.usb.prefix":"24","tethering.wifi.connectedClients":0,"tethering.wifi.dhcpserver.endip":"192.168.1.30","tethering.wifi.dhcpserver.startip":"192.168.1.10","tethering.wifi.enabled":false,"tethering.wifi.ip":"192.168.1.1","tethering.wifi.prefix":"24","tethering.wifi.security.password":"1234567890","tethering.wifi.security.type":"open","tethering.wifi.ssid":"FirefoxHotspot","time.clock.automatic-update.enabled":true,"time.timezone":"America/Los_Angeles","time.timezone.automatic-update.enabled":true,"time.timezone.user-selected":"America/Los_Angeles","ums.enabled":false,"ums.mode":0,"ums.status":0,"ums.volume.extsdcard.enabled":false,"ums.volume.sdcard.enabled":true,"vibration.enabled":true,"wallpaper.image":{"type":"image/jpeg","size":125849},"wap.push.enabled":true,"wifi.connect_via_settings":false,"wifi.disabled_by_wakelock":false,"wifi.enabled":true,"wifi.notification":false,"wifi.screen_off_timeout":600000,"wifi.suspended":false}}, {e88c87e9-0c19-4cfd-bccc-f7e37258544d}, {e88c87e9-0c19-4cfd-bccc-f7e37258544d} 05:24:07 ERROR - Return code: 256 05:24:07 INFO - gecko.log not found 05:24:07 INFO - TinderboxPrint: marionette: <em class="testfail">T-FAIL</em> 05:24:07 ERROR - Marionette exited with return code 256: harness failures 05:24:07 ERROR - # TBPL FAILURE #

Apparently there is a big divergence in runtests.py between v1.2 and master. git diff upstream/v1.2...upstream/master -- tests/python/gaia-ui-tests/gaiatest/runtests.py

We should probably lift all of Marionette's runtests.py changes from m-c to mozilla-b2g26, and likewise gaiatest's runtests.py changes from master to v1.2.

We need some uplifts from mozilla-central.

remote: https://hg.mozilla.org/releases/mozilla-b2g26_v1_2/rev/3141fa09989e remote: https://hg.mozilla.org/releases/mozilla-b2g26_v1_2/rev/11c08b521d97 remote: https://hg.mozilla.org/releases/mozilla-b2g26_v1_2/rev/ee4448b8832a remote: https://hg.mozilla.org/releases/mozilla-b2g26_v1_2/rev/6d86d0bb756e hopefully this will fix it.

fyi, if this "works", it won't actually cause the tests to pass, it will simply let us see what the real cause of failure is.

Indeed, that gives more useful information. https://tbpl.mozilla.org/php/getParsedLog.php?id=29985614&tree=Mozilla-B2g26-v1.2 12:19:19 INFO - ====================================================================== 12:19:19 INFO - ERROR: None 12:19:19 INFO - ---------------------------------------------------------------------- 12:19:19 INFO - Traceback (most recent call last): 12:19:19 INFO - File "/builds/slave/test/build/venv/local/lib/python2.7/site-packages/marionette/marionette_test.py", line 143, in run 12:19:19 INFO - testMethod() 12:19:19 INFO - File "/builds/slave/test/gaia/tests/python/gaia-ui-tests/gaiatest/tests/functional/settings/test_settings_change_language.py", line 22, in test_change_language_settings 12:19:19 INFO - self.wait_for_condition(lambda m: settings.header_text == u'Param\u00E8tres') 12:19:19 INFO - File "/builds/slave/test/build/venv/local/lib/python2.7/site-packages/gaiatest/gaia_test.py", line 662, in wait_for_condition 12:19:19 ERROR - raise TimeoutException(message) 12:19:19 ERROR - TEST-UNEXPECTED-FAIL | test_settings_change_language.py test_settings_change_language.TestChangeLanguage.test_change_language_settings | TimeoutException: Condition timed out 12:19:19 INFO - ---------------------------------------------------------------------- 12:19:19 INFO - Ran 1 test in 43.380s 12:19:19 WARNING - FAILED (errors=1) 12:20:29 INFO - ====================================================================== 12:20:29 INFO - FAIL: None 12:20:29 INFO - ---------------------------------------------------------------------- 12:20:29 INFO - Traceback (most recent call last): 12:20:29 INFO - File "/builds/slave/test/build/venv/local/lib/python2.7/site-packages/marionette/marionette_test.py", line 143, in run 12:20:29 INFO - testMethod() 12:20:29 INFO - File "/builds/slave/test/gaia/tests/python/gaia-ui-tests/gaiatest/tests/functional/ftu/test_ftu_with_tour.py", line 68, in test_ftu_with_tour 12:20:29 INFO - "Swipe down to access recent notifications, credit information and settings.") 12:20:29 INFO - TEST-UNEXPECTED-FAIL | test_ftu_with_tour.py test_ftu_with_tour.TestFtu.test_ftu_with_tour | AssertionError: u'Enter any keyword or topic and your phone will instantly adapt.' != 'Swipe down to access recent notifications, credit information and settings.' 12:20:29 INFO - ---------------------------------------------------------------------- 12:20:29 INFO - Ran 1 test in 16.214s 12:20:29 INFO - FAILED (failures=1)

I've disabled these on v1.2 so that we can reopen the tree: https://github.com/mozilla-b2g/gaia/commit/44ed9e129e6b6134dc3f2e32d930d740e4c1f746 I'll file separate bugs for these after confirming with a few retriggers.

This was resolved by the above uplifts; the actual test failures are covered separately by bug 933977 and bug 933980.