The main issue is the hardcoded 4444 port in head_update.js.in. I've been tackling this issue but I'm not sure how to fix it because of the way the files are loaded from one another (shared.js, head_update.js and the test files). Suggestions welcome!

There are only two app update xpcshell tests that need to be fixed - test_0030_general.js and test_0082_prompt_unsupportAlreadyNotified.js. The remainder now use mock implementations. http://mxr.mozilla.org/mozilla-central/source/toolkit/mozapps/update/test/unit/test_0030_general.js#32 http://mxr.mozilla.org/mozilla-central/source/toolkit/mozapps/update/test/unit/test_0082_prompt_unsupportAlreadyNotified.js#19 I'll see what we can do for these remaining two.

(In reply to Robert Strong [:rstrong] (do not email) from comment #1) > There are only two app update xpcshell tests that need to be fixed - > test_0030_general.js and test_0082_prompt_unsupportAlreadyNotified.js. The > remainder now use mock implementations. I was wondering why only so few tests actually start the HTTP Server. > http://mxr.mozilla.org/mozilla-central/source/toolkit/mozapps/update/test/ > unit/test_0030_general.js#32 > > http://mxr.mozilla.org/mozilla-central/source/toolkit/mozapps/update/test/ > unit/test_0082_prompt_unsupportAlreadyNotified.js#19 > > I'll see what we can do for these remaining two. Great, thanks. Please make sure that the tests pass with the patch in bug 887054.

:rstrong, I tried skipping those two tests and running the others concurrently and I still get failures.

please provide try links to the logs or the logs for these failures

needinfo on comment #4 Also, app update has many cases where it has to work with an install dir associated path. When running concurrently are you using the same <path>/xpcshell binary? We might be able to work around this if it is the case though it will take a decent amount of work and make many of the tests run longer since we will have to copy the bin dir for more tests.

(In reply to Robert Strong [:rstrong] (do not email) from comment #5) > needinfo on comment #4 > > Also, app update has many cases where it has to work with an install dir > associated path. When running concurrently are you using the same > <path>/xpcshell binary? We might be able to work around this if it is the > case though it will take a decent amount of work and make many of the tests > run longer since we will have to copy the bin dir for more tests. The binary dir is the same, yes. Every test gets its own profile dir. Can't the tests share the binaries and store the actual data inside the profile dirs?

(In reply to Mihnea Dobrescu-Balaur (:mihneadb) from comment #6) > (In reply to Robert Strong [:rstrong] (do not email) from comment #5) > > needinfo on comment #4 > > > > Also, app update has many cases where it has to work with an install dir > > associated path. When running concurrently are you using the same > > <path>/xpcshell binary? We might be able to work around this if it is the > > case though it will take a decent amount of work and make many of the tests > > run longer since we will have to copy the bin dir for more tests. > > The binary dir is the same, yes. Every test gets its own profile dir. > > Can't the tests share the binaries and store the actual data inside the > profile dirs? In the cases where it can it already does and there are numerous tests where it can't. App update is install dir specific and not mozilla profile specific when it comes to updating install dir files. I can likely copy the bin dir in those cases where we can't specify a different dir as mentioned before.

(In reply to Robert Strong [:rstrong] (do not email) from comment #7) > (In reply to Mihnea Dobrescu-Balaur (:mihneadb) from comment #6) > > (In reply to Robert Strong [:rstrong] (do not email) from comment #5) > > > needinfo on comment #4 > > > > > > Also, app update has many cases where it has to work with an install dir > > > associated path. When running concurrently are you using the same > > > <path>/xpcshell binary? We might be able to work around this if it is the > > > case though it will take a decent amount of work and make many of the tests > > > run longer since we will have to copy the bin dir for more tests. > > > > The binary dir is the same, yes. Every test gets its own profile dir. > > > > Can't the tests share the binaries and store the actual data inside the > > profile dirs? > In the cases where it can it already does and there are numerous tests where > it can't. App update is install dir specific and not mozilla profile > specific when it comes to updating install dir files. I can likely copy the > bin dir in those cases where we can't specify a different dir as mentioned > before. I'm also adding an option for the ini files to run particular tests sequentially. Not sure if that would be better/faster than copying the files?

It might very well be. I'll evaluate that as I investigate

(In reply to Mihnea Dobrescu-Balaur (:mihneadb) from comment #8) > (In reply to Robert Strong [:rstrong] (do not email) from comment #7) > > (In reply to Mihnea Dobrescu-Balaur (:mihneadb) from comment #6) > > > (In reply to Robert Strong [:rstrong] (do not email) from comment #5) > > > > needinfo on comment #4 > > > > > > > > Also, app update has many cases where it has to work with an install dir > > > > associated path. When running concurrently are you using the same > > > > <path>/xpcshell binary? We might be able to work around this if it is the > > > > case though it will take a decent amount of work and make many of the tests > > > > run longer since we will have to copy the bin dir for more tests. > > > > > > The binary dir is the same, yes. Every test gets its own profile dir. > > > > > > Can't the tests share the binaries and store the actual data inside the > > > profile dirs? > > In the cases where it can it already does and there are numerous tests where > > it can't. App update is install dir specific and not mozilla profile > > specific when it comes to updating install dir files. I can likely copy the > > bin dir in those cases where we can't specify a different dir as mentioned > > before. > > I'm also adding an option for the ini files to run particular tests > sequentially. Not sure if that would be better/faster than copying the files? Another issue is that a significant number of the tests launch the application (Firefox, Thunderbird, SeaMonkey, and B2G) itself to verify some of the startup operations that aren't available to xpcshell tests or any of the other harnesses. This could easily slow the system down to a crawl due to multiple instances of the application. I think I might be able to get test_0030_general.js and test_0082_prompt_unsupportAlreadyNotified.js to run parallel but I think it would be best to run test_0000_bootstrap_svc.js and all tests starting at test_0110* and above sequentially.

Your guess is better than mine here, so if you want to do that, just add run-sequentially = <REASON> in the manifest file for those tests that need that. I'm marking bug 887054 as a dependency because of this.

Note: one of the problems is on windows we use a mutex based on the application's path to prevent other instances from updating when an instance is updating. Need to figure out a decent workaround for this

Restoring summary since this applies to almost all of the tests.

Tests passing when run parallel (toolkit/mozapps/update/test/unit): test_0010_general.js test_0020_general.js test_0030_general.js test_0040_general.js test_0050_general.js test_0060_manager.js test_0061_manager.js test_0062_manager.js test_0063_manager.js test_0064_manager.js test_0070_update_dir_cleanup.js test_0071_update_dir_cleanup.js test_0072_update_dir_cleanup.js test_0073_update_dir_cleanup.js test_0080_prompt_silent.js test_0081_prompt_uiAlreadyOpen.js test_0082_prompt_unsupportAlreadyNotified.js test_bug595059.js test_bug794211.js test_bug833708.js test_0110_general.js test_0111_general.js test_0112_general.js test_0113_general.js test_0114_general.js test_0114_productChannelCheck.js test_0115_general.js test_0150_appBinReplaced_xp_win_complete.js test_0151_appBinPatched_xp_win_partial.js test_0152_appBinReplaced_xp_win_complete.js test_0153_appBinPatched_xp_win_partial.js test_0160_appInUse_complete.js test_0161_appInUse_xp_unix_complete.js test_0161_appInUse_xp_win_complete.js test_0162_appInUse_xp_win_complete.js test_0170_fileLocked_xp_win_complete.js test_0171_fileLocked_xp_win_partial.js test_0172_fileLocked_xp_win_complete.js test_0173_fileLocked_xp_win_partial.js test_0174_fileLocked_xp_win_complete.js test_0175_fileLocked_xp_win_partial.js test_0180_fileInUse_xp_win_complete.js test_0181_fileInUse_xp_win_partial.js test_0182_rmrfdirFileInUse_xp_win_complete.js test_0183_rmrfdirFileInUse_xp_win_partial.js test_0184_fileInUse_xp_win_complete.js test_0185_fileInUse_xp_win_partial.js test_0186_rmrfdirFileInUse_xp_win_complete.js test_0187_rmrfdirFileInUse_xp_win_partial.js test_0188_fileInUse_xp_win_complete.js test_0189_fileInUse_xp_win_partial.js test_0190_rmrfdirFileInUse_xp_win_complete.js test_0191_rmrfdirFileInUse_xp_win_partial.js toolkit/mozapps/update/test_svc/unit All of the tests in this dir will need to run sequentially due to testing the maintenance service To Do toolkit/mozapps/update/test/unit needs to use a dynamic port: * test_0030_general.js * test_0082_prompt_unsupportAlreadyNotified.js failing (haven't investigated) test_0113_versionDowngradeCheck.js to be evaluated (possible candidates to run sequentially) test_0200_app_launch_apply_update.js test_0201_app_launch_apply_update.js skip-if = os == 'win' [test_0202_app_launch_apply_update_dirlocked.js] skip-if = os == 'win' [test_0203_app_launch_apply_update.js] skip-if = os == 'win' [test_0300_update_root_dir_migration.js]

Brian, this gets us most of the way there. I think I can get the tests to always use adjustGeneralPaths but I'd like to do that in a general cleanup bug and not here so this bug isn't held up.

Try push https://tbpl.mozilla.org/?tree=Try&rev=3c21bc14d32e

Comment on attachment 776148 [details] [diff] [review] 1. use custom XREExeF and UpdRootD Review of attachment 776148 [details] [diff] [review]: ----------------------------------------------------------------- ::: toolkit/mozapps/update/test/unit/head_update.js.in @@ +2169,5 @@ > + let dirProvider = { > + getFile: function DP_getFile(prop, persistent) { > + persistent.value = true; > + if (prop == XRE_EXECUTABLE_FILE) > + return do_get_file(getApplyDirPath() + "test.bin", true); + "test." + APP_BIN_SUFFIX, here and below ? ::: toolkit/mozapps/update/test/unit/test_bug833708.js @@ +23,4 @@ > } > }; > > function run_test() { I don't think this test uses getApplyDirFile, but you may want to add TEST_ID here and anywhere else missing for consistency and future usage.

Carrying forward r+ Pushed to try again (original try push looked good but the results went away with the reset) https://tbpl.mozilla.org/?tree=Try&rev=376277bd268d

Comment on attachment 776830 [details] [diff] [review] 2. add run-sequentially for tests that can't run in parallel yet. Review of attachment 776830 [details] [diff] [review]: ----------------------------------------------------------------- thanks!

Pushed to mozilla-inbound https://hg.mozilla.org/integration/mozilla-inbound/rev/f8600c4facba https://hg.mozilla.org/integration/mozilla-inbound/rev/3ad86fe085e8

Also a followup test fix https://hg.mozilla.org/integration/mozilla-inbound/rev/82b1b2c74e2b

https://hg.mozilla.org/mozilla-central/rev/f8600c4facba https://hg.mozilla.org/mozilla-central/rev/3ad86fe085e8 https://hg.mozilla.org/mozilla-central/rev/82b1b2c74e2b

Hi, just encountered this failure in one of my try runs. http://pastebin.mozilla.org/2807096 Any ideas? (obviously the exe is being used, what other tests use it? I'll check myself but maybe you already know)

Worth mentioning that the test was indeed in the "run sequentially" part so it was alone.

No idea