With bug 1047728 the app bundles on OS X will be signed with a new signing method Apple introduces with 10.9.5. As result the channel-prefs.js file, which we use for changing the update channel gets moved to Content/Resources on OS X. That means the steps for updates will change and we have to do the following: 1. Check the app bundle signature. For that I have filed https://github.com/mozilla/mozmill-automation/issues/163 2. Firefox has to be started once to clear the quarantine bit. Only with that done we can modify the channel-prefs.js file. Given that we can't return to Python yet in between restarts, the file has to be modified directly while Firefox is running. For that we can use "GreD", which has been modified to point to Content/Resources. For Windows and Linux we might have to find another constant. Keep in mind that with the file being modified, we cannot check the signature again. 3. Perform the update steps as necessary (partial/full + optional fallback) 4. Revert the channel-prefs.js file The above only covers the tests. For the update-channel.js modification changes on the Python side I filed https://github.com/mozilla/mozmill-automation/issues/164

It would be great to add a step 5: 5. Check the new signature on the updated bundle (after step 4 was performed, of course).

Sure. I actually missed that. Checking the signatures will happen in a separate step. First we will complete the update tasks. Here a comment from Stephen Pohl [:spohl] from bug 1047728 comment #24: > > Here you go: http://people.mozilla.org/~spohl/FirefoxNightly.app.zip > > A couple things you may try: > codesign -vvvv FirefoxNightly.app -> This should tell you that the bundle > signature is valid. > codesign -dv FirefoxNightly.app -> This should tell you that the signature > version is 2 (second to last line) > spctl -a -v FirefoxNightly.app -> This should tell you that the app is > accepted based on the Developer ID. > > I may state the obvious, but remember that Gatekeeper will only verify the > signature the first time the app is launched. You may want to download it > and make a copy before you're testing things. You can then delete the > already launched app and replace it with the copy, which will still have the > quarantine bit set. Sure. Our update testrun always makes a copy of the binary before doing any modifications.

Ok, so I thought that I would like to rewrite the update tests first so all is in a single test module. But after thinking more about it, we should move this out and do it afterward. Getting this feature in is way more important.

This patch has not been tested with a test build on OS X yet, but it works fine on Linux. So lets see if the test build works as expected.

The test build I mentioned in my last comment can be found here: http://people.mozilla.org/~spohl/FirefoxNightly.app.zip

I have issues by running Mozmill with the app bundle of this test build on OS X. So I requested a DMG from Stephen. It will be available at http://people.mozilla.org/~spohl/FirefoxNightly.dmg soon.

Comment on attachment 8482778 [details] [diff] [review] module changes + channel-pref.js v1 Review of attachment 8482778 [details] [diff] [review]: ----------------------------------------------------------------- I tested this patch with existing builds on Linux and OS X so far. All is fine. So we can already let the test modify the channel-pref.js file. I will have to check how it works for Windows, but I don't think we will regress something. For now please have a look at this patch.

Comment on attachment 8482778 [details] [diff] [review] module changes + channel-pref.js v1 It doesn't look to be complete yet. We also might have to modify the file update-settings.ini in our tests. Anyway I would add this as a second patch, which has to be landed later. We would require an update of the mozmill-automation package first.

Comment on attachment 8482778 [details] [diff] [review] module changes + channel-pref.js v1 Review of attachment 8482778 [details] [diff] [review]: ----------------------------------------------------------------- I noticed that we run into problems when no --channel parameter gets specified. I will have to re-work this patch a bit.

This fixes the problem when no --channel option has been specified. Please note that I made some refactoring for a better handling of the default update channel settings.

Comment on attachment 8483468 [details] [diff] [review] module changes + channel-pref.js v2 Review of attachment 8483468 [details] [diff] [review]: ----------------------------------------------------------------- I believe we missed one file here: http://hg.mozilla.org/qa/mozmill-tests/file/default/firefox/tests/update/testFallbackUpdate/test3.js#l48 With that change, it tests well on Windows too.

Ups, good catch. I missed a final qref. :) Updated patch coming up in a bit.

Updated patch with the included test3.js for fallback updates

Comment on attachment 8483536 [details] [diff] [review] module changes + channel-pref.js v3 Review of attachment 8483536 [details] [diff] [review]: ----------------------------------------------------------------- Looks good now.

https://hg.mozilla.org/qa/mozmill-tests/rev/faf8565c06e6 (master) All branches would need this patch. I cannot flag esr24 anymore. Not sure when builds are gone, but we should even land it there. Daniel, please have a special look at those tests today. If all is fine I would like to get it landed on aurora later today.

Reports: http://mozmill-daily.blargon7.com/#/update/reports?app=All&branch=35.0&platform=All&from=2014-09-04&to=2014-09-04 So besides the unstable updates (bug 1045072) everything went fine for Linux & Mac for today.

All update tests on Windows succeeded. So I pushed the changes to aurora: https://hg.mozilla.org/qa/mozmill-tests/rev/dc594743f37f

https://hg.mozilla.org/qa/mozmill-tests/rev/3c81ea9e5913 (beta) The patch fails to apply cleanly for release because of the non-finished work on bug 883120. So I will have to update the patch.

Comment on attachment 8484973 [details] [diff] [review] module changes + channel-pref.js (backport release) Review of attachment 8484973 [details] [diff] [review]: ----------------------------------------------------------------- http://hg.mozilla.org/qa/mozmill-tests/rev/0ce961774c40 (release)

Thanks for landing Andreea, but please let me land my patches myself in the future, unless requested otherwise. Also landed on esr31: https://hg.mozilla.org/qa/mozmill-tests/rev/c49a0f5aef8c So we are done here. I will file another bug for the necessary update-settings.ini changes.

(In reply to Henrik Skupin (:whimboo) from comment #21) > So we are done here. I will file another bug for the necessary > update-settings.ini changes. This is actually bug 1063584.

Just for reference the patch for esr24 need some lines to be changed due to merge conflicts. Nothing else. Given that this is a bustage fix, lets get it in immediately. https://hg.mozilla.org/qa/mozmill-tests/rev/f068c5abcbb4 (esr24)