Given by the amount of work we have each time to run the update testing on betatest, beta, releasetest, and release channel we should have a neat Mozmill script which covers partial/complete/fallback upgrades. My idea is the following: 1. Have a Python script in-place which extracts a local build (dmg, zip, tar) to a temporary location and updates the channel info. It will receive two params: a) the build location and b) the channel. After those action has been done it calls the mozmill test from the command line. 2. The Mozmill script has to be implemented as a restart test. It will receive the extracted build and the path to the partial/complete update test directory. There will be two tests: a) Starts the update via the help menu and restarts the browser, b) checks if the version has been upgraded (blocked by bug 500987) after the upgrade. 3. For fallback updates we can run prepare the build a second time (see step 1) and call another mozmill test for the fallback part. 4. That Mozmill script has to be a restart test too. It will be started the same way as in step 2. There are also two tests: a) Starts the update via the help menu and modifies the update.status/update.mar file before the restart, b) checks if the version has been upgraded after the restart. Comments and hints are welcome.

Step 4 could be modified to account for a more accurate way in which updates could fail for users, other than editing the update.status/update.mar files.

So what do you think about? I'm not sure how it will work at all. I would be happy to have the first implementation ready. And as give by Robert via an email on the qa mailing list modifying the update.mar file could be the first step.

Mikeal, didn't you have something for updating testing in the works?

(In reply to comment #1) > Step 4 could be modified to account for a more accurate way in which updates > could fail for users, other than editing the update.status/update.mar files. The most useful test this could provide would be for the case where it fails to apply a partial mar and is initiated by Firefox starting up. The most likely scenario would be where the crc for a file does not match (e.g. it doesn't match the original file). Though it is possible that the mar file could be corrupted if the mar file was verified to apply properly before deployment this would be checked immediately upon download by the file hash check.

Thanks Robert. So I'll go the way with modifying the update.mar file to produce a broken CRC check. Can you please specify which folders I have to check for the update? It can be downloaded under the application folder and under local settings, right? Is there a constant for the latter folder available?

This test will cover the following existent Litmus tests: 6145: No Updates Found 6661: Apply software update should display Restart Confirmation 6673: Software update dialog wait for idle before prompting

(In reply to comment #5) > Thanks Robert. So I'll go the way with modifying the update.mar file to produce > a broken CRC check. Can you please specify which folders I have to check for > the update? It can be downloaded under the application folder and under local > settings, right? Is there a constant for the latter folder available? It would be simpler and more realistic if the crashreporter binary was renamed for the test.

(In reply to comment #7) > It would be simpler and more realistic if the crashreporter binary was renamed > for the test. So the following steps would be sufficient? 1. Download the partial/complete update 2. Rename the crashreporter binary 3. Restart Firefox

Yes. Also, 1 and 2 can be swapped or done at the same time if you prefer.

Robert, is it expected that I cannot create an instance of nsIApplicationUpdateService when the user hasn't sufficient permissions? I noticed that when I wanted to compare the menu item state with the aus.canUpdate property. When I click on the "Check for updates" item directly before opening the menu I can download an update. Successive tries are prohibited. Does it mean that the update service gets unregistered?

(In reply to comment #10) > Robert, is it expected that I cannot create an instance of > nsIApplicationUpdateService when the user hasn't sufficient permissions? I That was my fault. I was able to fix it. > noticed that when I wanted to compare the menu item state with the > aus.canUpdate property. When I click on the "Check for updates" item directly > before opening the menu I can download an update. Successive tries are > prohibited. Does it mean that the update service gets unregistered? Due to a limitation in our menuAPI for Mozmill I'm not able to click the help menu itself. So the update menu item is not initialized correctly. Looks like in such a case I have to check aus.canUpdate only.

Before I upload a test in the next days I have a screencast ready. See http://screencast.com/t/K668N0rF

This might be a separate bug but it would be _awesome_ if you could do the following (starting off with 3.5.1 in this example) * Update 3.5.1 -> 3.5.2 * Modify app.update.url.override to point back to the 3.5.1 snippets * Check for updates again This would let us test the client side updater code *before* shipping, which would be superb.

(In reply to comment #12) > Before I upload a test in the next days I have a screencast ready. See > http://screencast.com/t/K668N0rF Where's the script can you attach a WIP to the bug ?

(In reply to comment #8) > So the following steps would be sufficient? > > 1. Download the partial/complete update > 2. Rename the crashreporter binary > 3. Restart Firefox Rob, those steps will not work for Firefox 3.0.x. Removing the crash reporter doesn't trigger a fallback update. It will never be recreated. Is that known or shall I file a new bug on that?

Filed bug 507856.

Try a different file... perhaps the null plugin

(In reply to comment #17) > Try a different file... perhaps the null plugin It will not work for full updates. It will simply be replaced. I really have to make the update.mar file invalid. Rob, is there a simple way to retrieve the folder where the update has been downloaded to? (In reply to comment #13) > This might be a separate bug but it would be _awesome_ if you could do the > following (starting off with 3.5.1 in this example) > * Update 3.5.1 -> 3.5.2 > * Modify app.update.url.override to point back to the 3.5.1 snippets > * Check for updates again > > This would let us test the client side updater code *before* shipping, which > would be superb. Ben, it would be great when you can offer some more details of the purpose of this extra test. What can we test with this extra path?

(In reply to comment #18) > Ben, it would be great when you can offer some more details of the purpose of > this extra test. What can we test with this extra path? It allows us to test the ability of a build to update. At a couple points in the 3.5 release cycle we hit updater bugs that only occurred after we shipped the *next* version. For example, we hit a bug in 3.5b4 that was only found in when we shipped 3.5b99.

Rob, I need some help to figure out in which folder the update has been downloaded to. As what I have seen we use the update sub folder under the application directory if the application hasn't been installed to the default place. Otherwise it is under document and settings on Windows. All this is kinda complex. Do we have an easy way to get this folder?

This relies on the status file existing and should get the directory and the status file var fileLocator = Components.classes["@mozilla.org/file/directory_service;1"].getService(Components.interfaces.nsIProperties); var updateDir = fileLocator.get("XCurProcD", Components.interfaces.nsIFile); updateDir.append("updates"); updateDir.append("0"); var updateStatus = updateDir.clone(); updateStatus.append("update.status"); if (!updateStatus.exists()) { updateDir = fileLocator.get("UpdRootD", Components.interfaces.nsIFile); updateDir.append("updates"); updateDir.append("0"); updateStatus = updateDir.clone(); updateStatus.append("update.status"); }

Thanks Rob! That was a great help. For todays beta channel test I have only used the update folder within the application folder. I will check with the UpdRootD folder in a bit. Meanwhile I have a problem in determining what type of update I have. When asking the update service I always get a partial one returned: var umService = Cc["@mozilla.org/updates/update-manager;1"] .getService(Ci.nsIUpdateManager); var update = umService.activeUpdate; var complete = update.isCompleteUpdate; Shouldn't I get a true instead a false for 3.5=>3.5.3?

You would if the update snippet returned the correct information https://aus2.mozilla.org/update/3/Firefox/3.5/20090624025744/WINNT_x86-msvc/en-US/release/Windows_NT%206.0/default/default/update.xml?force=1 It returns both a partial and a complete with the same info which is bogus.

I have filed bug 514040 on that issue.

To workaround the update snippet specifying the same mar for both the partial and complete you could do something along the lines of the following to check if a complete will be offered after the partial which is the same as the complete var um = Components.classes["@mozilla.org/updates/update-manager;1"]. getService(Components.interfaces.nsIUpdateManager); var updates = um.getUpdateAt(0); var patch1 = updates.getPatchAt(0); var patch2 = updates.getPatchAt(1); if (patch1.URL == patch2.URL) alert("The Same");

(In reply to comment #25) > var updates = um.getUpdateAt(0); > var patch1 = updates.getPatchAt(0); > var patch2 = updates.getPatchAt(1); > if (patch1.URL == patch2.URL) > alert("The Same"); But that doesn't tell me that I have a partial or complete update, or do have complete updates only one patch in the update snippet?

Both, partial and complete updates have a patchCount of 2. :/

(In reply to comment #25) > var patch1 = updates.getPatchAt(0); > var patch2 = updates.getPatchAt(1); > if (patch1.URL == patch2.URL) > alert("The Same"); Ok, got it. In the case of a complete update both patches have the same URL. That will work until bug 514040 gets fixed. Thanks.

Exactly... an update can have 1 or 2 patches and the patches can either be partial or complete patches.

(In reply to comment #25) > var um = Components.classes["@mozilla.org/updates/update-manager;1"]. > getService(Components.interfaces.nsIUpdateManager); > var updates = um.getUpdateAt(0); Rob, I have another problem with the lines above. When I can access the current update? When I do it right after the update check the update at index 0 has the same buildID as my current build. When I try to use activeUpdate then null is returned. Shouldn't I have access to the update data before I start the download?

(In reply to comment #30) > (In reply to comment #25) > > var um = Components.classes["@mozilla.org/updates/update-manager;1"]. > > getService(Components.interfaces.nsIUpdateManager); > > var updates = um.getUpdateAt(0); > > Rob, I have another problem with the lines above. When I can access the current > update? When I do it right after the update check the update at index 0 has the > same buildID as my current build. When I try to use activeUpdate then null is > returned. Shouldn't I have access to the update data before I start the > download? The update won't be active until the choice to download it has been made.

(In reply to comment #31) > The update won't be active until the choice to download it has been made. Ok, so do I have a chance to fetch it via the getUpdateAt function? It looks like that it is not in the list after the update check has been finished.

No. getUpdateAt is only available after the update has been selected (e.g. the choice to download it has been made).

This is a WIP which let you run minor software updates on all platforms. The Mozmill tests itself are mostly done but the Python backend has to be rewritten to use the Mozrunner CLI class which will simplify a lot of stuff and will give me access to the persisted object of Mozmill.

Robert, today I tried if the current software update tests also work for Firefox 3.0.x. Everything looks fine except one thing I would have to add for an additional check. Right now I check after the update if no other update is available. That works on 3.5 but fails for 3.0.14 because we offer a major update. So what I really want to know is if it is possible to retrieve the information that we have a minor/major update right after the check for updates has been performed. The software update dialog also displays the new version number and the the extra buttons so the update information should be available before starting the download. It would be really helpful when you could give us the information.

(In reply to comment #35) > Robert, today I tried if the current software update tests also work for > Firefox 3.0.x. Everything looks fine except one thing I would have to add for > an additional check. Right now I check after the update if no other update is > available. That works on 3.5 but fails for 3.0.14 because we offer a major > update. So what I really want to know is if it is possible to retrieve the > information that we have a minor/major update right after the check for updates > has been performed. The software update dialog also displays the new version > number and the the extra buttons so the update information should be available > before starting the download. > > It would be really helpful when you could give us the information. Are you asking if it is possible to find out if the update is a major or minor update after you have performed a check for updates?

(In reply to comment #37) > Are you asking if it is possible to find out if the update is a major or minor > update after you have performed a check for updates? Correct. Right after I have opened the update dialog.

(In reply to comment #38) > (In reply to comment #37) > > Are you asking if it is possible to find out if the update is a major or minor > > update after you have performed a check for updates? > > Correct. Right after I have opened the update dialog. When I use the getUpdateAt() function I only see already applied updates but not the one which is in the queue right now.

I have a couple of other items I need to finish up before I will get to this... at the latest I should be able to look at this tomorrow.

(In reply to comment #35) > Robert, today I tried if the current software update tests also work for > Firefox 3.0.x. Everything looks fine except one thing I would have to add for > an additional check. Right now I check after the update if no other update is > available. That works on 3.5 but fails for 3.0.14 because we offer a major > update. So what I really want to know is if it is possible to retrieve the > information that we have a minor/major update right after the check for updates > has been performed. The software update dialog also displays the new version > number and the the extra buttons so the update information should be available > before starting the download. Per comment #31 with more details - the update info is either passed to the software update dialog when notifying of an update or the software update dialog is given the update info as a nsIUpdateCheckListener when manually checking for updates. It is possible to get this info via the same code (minus the ui code) as used by the software update dialog here. http://mxr.mozilla.org/mozilla-central/source/toolkit/mozapps/update/content/updates.js#457

I noticed you discussing that you access elements / values in xul with mozmill and if this is the case the ui has an element with the value you are looking for... don't recall off the top of my head which one but it is trivial to find it with DOMi, etc.

Thanks Rob. I will have a check in the next days.

Slightly updated to match current work for shared modules. I hope that I will have time for the next step soon.

There was a problem on Linux while extracing the bz2 archive to a given folder.

Robert, I have a problem on Windows with the uninstaller. Calling it via subprocess.call from Python should normally wait until it has been finished. But that doesn't happen. It returns immediately. Do we have a special kind of uninstaller or do we launch another application internally so the created child process finishes immediately?

It's probably just like Firefox in that it's a shell script that then launches another bin so the pid handler returns immediately.

The reason is that the uninstaller forks itself so the whole installation folder can be removed. Otherwise the file will locked. I will add a sleep and a check if the folder exists.

(In reply to comment #48) > The reason is that the uninstaller forks itself so the whole installation > folder can be removed. Otherwise the file will locked. I will add a sleep and a > check if the folder exists. Correct but note that it makes a copy of itself in temp so it is able to delete the install dir. You may want to check for the existence of the uninstall dir since if files are added by something other than the installer or updater the uninstaller won't try to remove them and the dir will still be there.

(In reply to comment #49) > Correct but note that it makes a copy of itself in temp so it is able to delete > the install dir. You may want to check for the existence of the uninstall dir > since if files are added by something other than the installer or updater the > uninstaller won't try to remove them and the dir will still be there. Thanks for the note. But can I make an assumption whether an installer or a zip build has been used to install Firefox on the local system? Both have the uninstall folder and I would really like to support both ways. If everything fails I could run a rmtree on the whole folder afterward.

We don't ship zip files for releases, they're unsupported. Someone would have to navigate into the likes of firefox/nightly/3.5.4-candidates/build3/unsigned/win32/<locale> to find a zip file.

That's correct but I create a general installer python class we will be able to use for different testing scenarios which will not only apply to release builds. Even update tests could be performed for nightly builds and sometimes it's easier/faster to just unzip a build instead of using the installer. I think about regression tests with Mozmill by downloading necessary builds.

zip builds can't be uninstalled using the uninstaller since they don't have an uninstall.log which can be used to determine whether it is a zip build or not.

Updated tests so they work on the latest mozmill updates

All remaining stuff has been done. Mikeal, could you please have a look at the Python back-end stuff? Aakash, a review for the tests would be great. What has been fixed in this version: * Fallback update was not possible in last version * If no channel has been specified it has not been reset for the next build * If the target folder for the installation has no trailing slash we failed * Uninstaller returned too fast due to a missing sleep call * If a major update is followed we ignore it gracefully * The test itself checks also for the correct channel A mozmill test for only major updates will follow later.

Comment on attachment 410545 [details] [diff] [review] Patch v1 Aakash, lemme call it ui-review. There is no way I can ask for another review.

Comment on attachment 410545 [details] [diff] [review] Patch v1 Henrik, just use addl. review

Some more updates which will make this version the final one: * I have included the major update checks. Therefor another command line parameter called "--type" can be used to specify a minor or major update check * The fallback update has been combined with the default update so both checks will be done per default. With the --no-fallback option you can exclude that check * A folder can be specified as installation source. In such a case all the files inside that folder will be processed. * Some more smaller fixes

Comment on attachment 411664 [details] [diff] [review] Patch v2 Mikeal, can you please check the Python backend changes? Thanks.

Comment on attachment 411664 [details] [diff] [review] Patch v2 Looks good to me. Great job, Henrik.

Yay! Tests have been landed as: http://hg.mozilla.org/qa/mozmill-tests/rev/4ad8c3954f13 http://hg.mozilla.org/qa/mozmill-tests/rev/b6c623260cf2 I'll update the documentation on QMO to match the latest changes.

Documentation for the SoftwareUpdateAPI has been added: https://developer.mozilla.org/en/Mozmill_Tests/Shared_Modules/SoftwareUpdateAPI

Mass move of Mozmill Test related project bugs to newly created components. You can filter out those emails by using "Mozmill-Tests-to-MozillaQA" as criteria.