10:17:13 INFO - Traceback (most recent call last): 10:17:13 INFO - File "/Users/mozauto/jenkins/workspace/mozilla-central_update/build/venv/lib/python2.7/site-packages/marionette/marionette_test.py", line 344, in run 10:17:13 INFO - testMethod() 10:17:13 INFO - File "/Users/mozauto/jenkins/workspace/mozilla-central_update/build/venv/lib/python2.7/site-packages/firefox_ui_tests/update/direct/test_direct_update.py", line 21, in test_update 10:17:13 INFO - self.check_update_applied() 10:17:13 INFO - File "/Users/mozauto/jenkins/workspace/mozilla-central_update/build/venv/lib/python2.7/site-packages/firefox_puppeteer/testcases/update.py", line 159, in check_update_applied 10:17:13 INFO - self.updates[self.current_update_index].type) Marionette also has thrown an error in takeScreenshot. Maybe its related. I will file that as a separate bug and will mark it as dependency.

This is failing a lot on the lastet beta update tests for Firefox 48.0: https://treeherder.mozilla.org/#/jobs?repo=mozilla-beta&revision=d142c49033c015f67272562b37dbe2912cfc7f14&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-beta-cdntest&selectedJob=1302898 I will check today what's causing that.

This problem seems to only occur for watershed releases when another update is in the queue. The following check will always fail because `type` is part of the `patch` sub-property: https://dxr.mozilla.org/mozilla-central/source/testing/firefox-ui/harness/firefox_ui_harness/testcases.py?q=file%3Atestcases.py&redirect_type=single#167

My watershed suspicion was correct. In the following example we started with a 46.0 release and tried to upgrade to latest beta. But we ended up with 48.0b1: https://aus5.mozilla.org/update/3/Firefox/46.0/20160411042519/WINNT_x86-msvc/en-US/beta-cdntest/Windows_NT%206.1/default/default/update.xml?force=1 Michelle, you should make sure not to include releases before the watershed release for testing on the given platform. Even with this bug fixed, the tests will fail - but then with a more helpful message.

Review commit: https://reviewboard.mozilla.org/r/63048/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/63048/

https://reviewboard.mozilla.org/r/63048/#review59976 ::: testing/firefox-ui/harness/firefox_ui_harness/testcases.py:208 (Diff revision 1) > + 'No further update of the same type gets offered: ' > + '{0} != {1}'.format( > + self.software_update.active_update.type, > + update['patch']['type'] > + )) > + I moved the whole block down here because it would be more important to do the version and buildid checks first before spitting out an update type failure, which is lesser understandable as why we upgraded to a specific version.

Comment on attachment 8769078 [details] Bug 1247596 - Fix reference to the update type which is part of the patch. https://reviewboard.mozilla.org/r/63048/#review59996

Pushed by hskupin@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/abb40d3751b5 Fix reference to the update type which is part of the patch. r=maja_zf

Maja, could you help getting this patch backported down to beta if all goes well by next week? It would just be adding the whiteboard entries. Thanks

This seems to be behaving well. As per Henrik's request, adding checkin-needed tags

(In reply to Henrik Skupin (:whimboo) from comment #6) > Michelle, you should make sure not to include releases before the watershed > release for testing on the given platform. Even with this bug fixed, the > tests will fail - but then with a more helpful message. Unfortunately we are mostly unaware when new watershed updates are setup. I've brought this up to Sylvestre so we can find a solution to get notified whenever new watersheds are setup, as it's not the first time that we find out about a new watershed after it was deployed.

Sounds great. Also the new failure message will clearly indicate now that Firefox has not been updated to the expected target build version but a former one.

(In reply to OrangeFactor Robot from comment #21) > 53 automation job failures were associated with this bug yesterday. > > Repository breakdown: > * mozilla-release: 53 Those latest test failures as listed by Orange Factor should not have been starred against this bug, but a new one filed. Given that this hasn't been done yesterday I will do it now.