User Agent: Mozilla/5.0 (Windows NT 6.1; rv:17.0) Gecko/20100101 Firefox/17.0 (Beta/Release) Build ID: 20100101 Steps to reproduce: I looked at different ESR24 builds done in Gitian build environments. Actual results: There were JavaScript files that got included differently into omni.ja files (see the attached binary diff of async.js as an example). Expected results: The omni.ja files should have been exactly the same across different builds done in Gitian environments.

While looking at mozjar.py and friends I was wondering how it could have happened in the first place that the path to the files is different in the compressed files (see the attachment where we have "resource://gre/modules/services-common/async.js" on one hand and "resource://services-common/async.js" on the other). It seems that there is already something wrong before the file got added to the omni.ja (and compressed), or am I wrong here?

It would be easier if you'd displayed the diff of the extracted omni.ja, because i'm pretty sure the file name for the differing file is something like jsloader/something, but it's impossible to tell from what you attached.

Well, a diff of the extracted omnj.ja just gives me: "Binärdateien gk/tor-browser_en-US/Browser/omni1/jsloader/resource/gre/modules/services-common/async.js und ln5/tor-browser_en-US/Browser/omni1/jsloader/resource/gre/modules/services-common/async.js sind verschieden." Then I did diff -u <(xxd gk/tor-browser_en-US/Browser/omni1/jsloader/resource/gre/modules/services-common/async.js) <(xxd ln5/tor-browser_en-US/Browser/omni1/jsloader/resource/gre/modules/services-common/async.js) > async_diff and attached async_diff. So, when I wrote "the path to the files is different in the compressed files" I meant something along the lines "the path to the files shown in the content of gk/tor-browser_en-US/Browser/omni1/jsloader/resource/gre/modules/services-common/async.js and ln5/tor-browser_en-US/Browser/omni1/jsloader/resource/gre/modules/services-common/async.js". Hope that helps...

So this is what i suspected. My guess is the problem comes from toolkit/mozapps/installer/precompile_cache.js using file-system file order when enumerating js files to precompile. The following patch should work, but its side effects on startup i/o patterns would have to be determined: diff --git a/toolkit/mozapps/installer/precompile_cache.js b/toolkit/mozapps/installer/precompile_cache.js --- a/toolkit/mozapps/installer/precompile_cache.js +++ b/toolkit/mozapps/installer/precompile_cache.js @@ -60,17 +60,17 @@ function get_modules_under(uri) { .concat(dir_entries(file.file, "modules", ".js")) .concat(dir_entries(file.file, "modules", ".jsm")); } else { throw "Expected a nsIJARURI or nsIFileURL"; } } function load_modules_under(spec, uri) { - var entries = get_modules_under(uri); + var entries = get_modules_under(uri).sort(); // The precompilation of JS here sometimes reports errors, which we don't // really care about. But if the errors are ever reported to xpcshell's // error reporter, it will cause it to return an error code, which will break // automation. Currently they won't be, because the component loader spins up // its JSContext before xpcshell has time to set its context callback (which // overrides the error reporter on all newly-created JSContexts). But as we // move towards a singled-cxed browser, we'll run into this. So let's be // forward-thinking and deal with it now.

Thanks for the patch, Mike! We gonna test it and I'll report back if it indeed fixes this issue.

FYI: This patch solves the problem.

Okay, here comes the workaround in patch form (against m-c). Please review.

Comment on attachment 8388471 [details] [diff] [review] Make packaging files into omni.ja deterministic. Review of attachment 8388471 [details] [diff] [review]: ----------------------------------------------------------------- I can't obviously review my own patch.