Filed by: nbeleuzu [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=273437334&repo=autoland
Full log: https://queue.taskcluster.net/v1/task/Cvc2UebrQg61rCIYXO49Zw/runs/0/artifacts/public/logs/live_backing.log

[task 2019-10-29T09:54:58.771Z] 09:54:58 INFO - TEST-START | browser/base/content/test/static/browser_all_files_referenced.js
[task 2019-10-29T09:55:05.428Z] 09:55:05 INFO - TEST-INFO | started process screencapture
[task 2019-10-29T09:55:05.617Z] 09:55:05 INFO - TEST-INFO | screencapture: exit 0
[task 2019-10-29T09:55:05.617Z] 09:55:05 INFO - Buffered messages logged at 09:54:58
[task 2019-10-29T09:55:05.617Z] 09:55:05 INFO - Entering test bound checkAllTheFiles
[task 2019-10-29T09:55:05.617Z] 09:55:05 INFO - Buffered messages logged at 09:55:05
[task 2019-10-29T09:55:05.618Z] 09:55:05 INFO - indirectly whitelisted file: chrome://marionette/content/test_dialog.dtd used from chrome://marionette/content/test_dialog.xul
[task 2019-10-29T09:55:05.618Z] 09:55:05 INFO - indirectly whitelisted file: chrome://global-platform/locale/intl.properties used from resource://gre/greprefs.js
[task 2019-10-29T09:55:05.618Z] 09:55:05 INFO - indirectly whitelisted file: resource://services-sync/addonutils.js used from resource://services-sync/engines/addons.js
[task 2019-10-29T09:55:05.619Z] 09:55:05 INFO - indirectly whitelisted file: resource://gre/modules/Promise-backend.js used from resource://gre/modules/Promise.jsm,resource://devtools/shared/worker/loader.js,resource://devtools/shared/Loader.jsm
[task 2019-10-29T09:55:05.619Z] 09:55:05 INFO - indirectly whitelisted file: resource://services-sync/engines/clients.js used from resource://services-sync/service.js
[task 2019-10-29T09:55:05.622Z] 09:55:05 INFO - indirectly whitelisted file: chrome://pippki/content/resetpassword.js used from chrome://pippki/content/resetpassword.xul
[task 2019-10-29T09:55:05.623Z] 09:55:05 INFO - indirectly whitelisted file: chrome://global/content/remote-test-ipc.js used from chrome://global/content/test-ipc.xul
[task 2019-10-29T09:55:05.623Z] 09:55:05 INFO - indirectly whitelisted file: resource://services-sync/stages/enginesync.js used from resource://services-sync/service.js
[task 2019-10-29T09:55:05.623Z] 09:55:05 INFO - indirectly whitelisted file: resource://services-sync/addonsreconciler.js used from resource://services-sync/engines/addons.js
[task 2019-10-29T09:55:05.624Z] 09:55:05 INFO - indirectly whitelisted file: resource://services-sync/collection_validator.js used from resource://services-sync/engines/addons.js,resource://services-sync/engines/forms.js,resource://services-sync/engines/passwords.js
[task 2019-10-29T09:55:05.624Z] 09:55:05 INFO - indirectly whitelisted file: resource://services-sync/policies.js used from resource://services-sync/service.js
[task 2019-10-29T09:55:05.624Z] 09:55:05 INFO - indirectly whitelisted file: resource://services-sync/bookmark_validator.js used from resource://services-sync/engines/bookmarks.js
[task 2019-10-29T09:55:05.624Z] 09:55:05 INFO - indirectly whitelisted file: resource://services-sync/collection_repair.js used from resource://services-sync/engines/clients.js,resource://services-sync/bookmark_repair.js,resource://services-sync/doctor.js
[task 2019-10-29T09:55:05.624Z] 09:55:05 INFO - indirectly whitelisted file: resource://services-sync/stages/declined.js used from resource://services-sync/service.js
[task 2019-10-29T09:55:05.625Z] 09:55:05 INFO - indirectly whitelisted file: chrome://pippki/content/load_device.js used from chrome://pippki/content/load_device.xul
[task 2019-10-29T09:55:05.627Z] 09:55:05 INFO - indirectly whitelisted file: resource://services-sync/engines.js used from resource://services-sync/engines/clients.js,resource://services-sync/engines/addons.js,resource://services-sync/engines/forms.js,resource://services-sync/engines/history.js,resource://services-sync/engines/tabs.js,resource://services-sync/engines/bookmarks.js,resource://services-sync/engines/extension-storage.js,resource://services-sync/engines/passwords.js,resource://services-sync/engines/prefs.js,resource://services-sync/service.js,resource://formautofill/FormAutofillSync.jsm
[task 2019-10-29T09:55:05.627Z] 09:55:05 INFO - indirectly whitelisted file: resource://services-common/logmanager.js used from resource://services-sync/policies.js
[task 2019-10-29T09:55:05.627Z] 09:55:05 INFO - indirectly whitelisted file: resource://gre/modules/Http.jsm used from resource://app/modules/translation/GoogleTranslator.jsm,resource://app/modules/translation/YandexTranslator.jsm,resource://app/modules/translation/BingTranslator.jsm
[task 2019-10-29T09:55:05.627Z] 09:55:05 INFO - indirectly whitelisted file: resource://gre/modules/SyncedBookmarksMirror.jsm used from resource://services-sync/engines/bookmarks.js
[task 2019-10-29T09:55:05.627Z] 09:55:05 INFO - indirectly whitelisted file: chrome://passwordmgr/content/recipes.json used from resource://gre/greprefs.js
[task 2019-10-29T09:55:05.629Z] 09:55:05 INFO - indirectly whitelisted file: resource://formautofill/FormAutofillSync.jsm used from resource://services-sync/service.js
[task 2019-10-29T09:55:05.629Z] 09:55:05 INFO - indirectly whitelisted file: resource://services-sync/bookmark_repair.js used from resource://services-sync/collection_repair.js
[task 2019-10-29T09:55:05.629Z] 09:55:05 INFO - indirectly whitelisted file: resource://services-sync/doctor.js used from resource://services-sync/bookmark_repair.js,resource://services-sync/stages/enginesync.js
[task 2019-10-29T09:55:05.630Z] 09:55:05 INFO - Buffered messages finished
[task 2019-10-29T09:55:05.633Z] 09:55:05 INFO - TEST-UNEXPECTED-FAIL | browser/base/content/test/static/browser_all_files_referenced.js | there should be no unreferenced files - Got 3, expected 0
[task 2019-10-29T09:55:05.633Z] 09:55:05 INFO - Stack trace:
[task 2019-10-29T09:55:05.633Z] 09:55:05 INFO - chrome://mochikit/content/browser-test.js:test_is:1312
[task 2019-10-29T09:55:05.633Z] 09:55:05 INFO - chrome://mochitests/content/browser/browser/base/content/test/static/browser_all_files_referenced.js:checkAllTheFiles:931
[task 2019-10-29T09:55:05.634Z] 09:55:05 INFO - chrome://mochikit/content/browser-test.js:Tester_execTest/<:1067
[task 2019-10-29T09:55:05.634Z] 09:55:05 INFO - chrome://mochikit/content/browser-test.js:Tester_execTest:1102
[task 2019-10-29T09:55:05.634Z] 09:55:05 INFO - chrome://mochikit/content/browser-test.js:nextTest/<:930
[task 2019-10-29T09:55:05.634Z] 09:55:05 INFO - chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:SimpleTest.waitForFocus/waitForFocusInner/focusedOrLoaded/<:805
[task 2019-10-29T09:55:05.634Z] 09:55:05 INFO - Not taking screenshot here: see the one that was previously logged

Henrik, could this be caused by the recent changes in Bug 1587627?

So my patch removes the entry for chrome://marionette/content/test_anonymous_content.xul because it has been removed from the jar.mn file and the repository.

The failures as complained about here for test.xul, test2.xul are strange, because all of those files are still being used by various Marionette unit tests. Also test_nested_iframe.xul is referenced in the above test.xul file.

Gijs, do you have an idea or could refer to someone who can help?

For now I will request a backout of the affecting patch over on bug 1587627.

Note that this only happens on MacOS and Windows, but not Linux:
https://treeherder.mozilla.org/#/jobs?repo=autoland&searchStr=bc6&fromchange=7a4cf39826594197b9bff4d988f49d4185ac759b&selectedJob=273437334

(In reply to Henrik Skupin (:whimboo) [⌚️UTC+2] from comment #3)

Note that this only happens on MacOS and Windows, but not Linux:
https://treeherder.mozilla.org/#/jobs?repo=autoland&searchStr=bc6&fromchange=7a4cf39826594197b9bff4d988f49d4185ac759b&selectedJob=273437334

Linux opt (non-debug, non-asan) mochitests just didn't run.

Gijs, sorry but my needinfo was more related to comment 2. I should have mentioned that.

Oh, no, they did, and they were orange:

https://treeherder.mozilla.org/#/jobs?repo=autoland&searchStr=linux%2C64%2Copt%2Cbc&revision=79be8382d57a67860918b238e2ddc47043d523ea&selectedJob=273442063

(In reply to Henrik Skupin (:whimboo) [⌚️UTC+2] from comment #2)

So my patch removes the entry for chrome://marionette/content/test_anonymous_content.xul because it has been removed from the jar.mn file and the repository.

This bit is fine, this file is not mentioned anywhere in the error output.

The failures as complained about here for test.xul, test2.xul are strange, because all of those files are still being used by various Marionette unit tests. Also test_nested_iframe.xul is referenced in the above test.xul file.

I actually think the error messages are fairly clear:

[task 2019-10-29T10:33:34.020Z] 10:33:34 INFO - TEST-UNEXPECTED-FAIL | browser/base/content/test/static/browser_all_files_referenced.js | there should be no unreferenced files - Got 3, expected 0
[task 2019-10-29T10:33:34.027Z] 10:33:34 INFO - TEST-UNEXPECTED-FAIL | browser/base/content/test/static/browser_all_files_referenced.js | unreferenced file: chrome://marionette/content/test.xul -
[task 2019-10-29T10:33:34.037Z] 10:33:34 INFO - TEST-UNEXPECTED-FAIL | browser/base/content/test/static/browser_all_files_referenced.js | file only referenced from unreferenced files: chrome://marionette/content/test2.xul referenced from chrome://marionette/content/test.xul,chrome://marionette/content/test_nested_iframe.xul -
[task 2019-10-29T10:33:34.043Z] 10:33:34 INFO - TEST-UNEXPECTED-FAIL | browser/base/content/test/static/browser_all_files_referenced.js | file only referenced from unreferenced files: chrome://marionette/content/test_nested_iframe.xul referenced from chrome://marionette/content/test.xul

So, as far as the test is concerned, nothing references test.xul, and test2.xul and test_nested_iframe.xul are only referenced from other files that aren't referenced (test.xul and test2.xul itself).

The right fix for this is to not package these files into Firefox, and pack them into the test framework instead.

(In reply to :Gijs (he/him) from comment #7)

So, as far as the test is concerned, nothing references test.xul, and test2.xul and test_nested_iframe.xul are only referenced from other files that aren't referenced (test.xul and test2.xul itself).

The same is the case for test_anonymous_content.xul. Which was also only referenced from a Marionette unit test. Why didn't it fail when it was still part of marionette (before my patch) and packaged with Firefox? I think that is the part I actually do not understand yet.

Maybe I will just leave the file in the repository for now and we will take care of it once we can work on bug 1344267.

Oh, I think that I got it now. I will add test.xul to the browser_all_files_referenced.js file, which should fix it. Thanks for your help.

Also the test failures should be gone now with the backout of the patch on bug 1587627.

Thank you Henrik for the quick response!