+++ This bug was initially created as a clone of Bug #1596962 +++

Bug 1568063 tracks intermittent mass xpcshell failures on Android: On some test runs, all xpcshell runs fail.

Some (probably a minority) of those task failures show xpcshell repeatedly crashing on startup with this assertion failure.

I previously tried to fix this in bug 1596962, but that was not effective. (But I'll leave the landed patch, since it is harmless.)

https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=277646836&repo=autoland&lineNumber=1396

[task 2019-11-22T13:28:15.786Z] 13:28:15     INFO -  TEST-START | caps/tests/unit/test_origin.js
[task 2019-11-22T13:28:16.394Z] 13:28:16  WARNING -  TEST-UNEXPECTED-FAIL | caps/tests/unit/test_origin.js | xpcshell return code: 139
[task 2019-11-22T13:28:16.394Z] 13:28:16     INFO -  TEST-INFO took 608ms
[task 2019-11-22T13:28:16.394Z] 13:28:16     INFO -  >>>>>>>
[task 2019-11-22T13:28:16.394Z] 13:28:16     INFO -  caps/tests/unit/test_origin.js | xpcw: cd /sdcard/tests/xpc/caps/tests/unit
[task 2019-11-22T13:28:16.394Z] 13:28:16     INFO -  caps/tests/unit/test_origin.js | xpcw: xpcshell -r /sdcard/tests/xpc/c/httpd.manifest --greomni /data/local/xpcb/geckoview-androidTest.apk -m -s -e const _HEAD_JS_PATH = "/sdcard/tests/xpc/head.js"; -e const _MOZINFO_JS_PATH = "/sdcard/tests/xpc/p/mozinfo.json"; -e const _PREFS_FILE = "/sdcard/tests/xpc/user.js"; -e const _TESTING_MODULES_DIR = "/sdcard/tests/xpc/m"; -f /sdcard/tests/xpc/head.js -e const _HEAD_FILES = []; -e const _JSDEBUGGER_PORT = 0; -e const _TEST_FILE = ["test_origin.js"]; -e const _TEST_NAME = "caps/tests/unit/test_origin.js"; -e _execute_test(); quit(0);
[task 2019-11-22T13:28:16.395Z] 13:28:16     INFO -  caps/tests/unit/test_origin.js | [2957, Unnamed thread 79014ed21020] WARNING: XPCOM objects created/destroyed from static ctor/dtor: file /builds/worker/workspace/build/src/xpcom/base/nsTraceRefcnt.cpp, line 198
[task 2019-11-22T13:28:16.395Z] 13:28:16     INFO -  caps/tests/unit/test_origin.js | [2957, Unnamed thread 79014ed21020] WARNING: XPCOM objects created/destroyed from static ctor/dtor: file /builds/worker/workspace/build/src/xpcom/base/nsTraceRefcnt.cpp, line 198
[task 2019-11-22T13:28:16.395Z] 13:28:16     INFO -  caps/tests/unit/test_origin.js | [2957, Unnamed thread 79014ed21020] WARNING: XPCOM objects created/destroyed from static ctor/dtor: file /builds/worker/workspace/build/src/xpcom/base/nsTraceRefcnt.cpp, line 198
[task 2019-11-22T13:28:16.395Z] 13:28:16     INFO -  caps/tests/unit/test_origin.js | [2957, Unnamed thread 79014ed21020] WARNING: XPCOM objects created/destroyed from static ctor/dtor: file /builds/worker/workspace/build/src/xpcom/base/nsTraceRefcnt.cpp, line 198
[task 2019-11-22T13:28:16.395Z] 13:28:16     INFO -  caps/tests/unit/test_origin.js | [2957, Main Thread] WARNING: No Android crash handler set: file /builds/worker/workspace/build/src/toolkit/crashreporter/nsExceptionHandler.cpp, line 1549
[task 2019-11-22T13:28:16.396Z] 13:28:16     INFO -  caps/tests/unit/test_origin.js | [2957, Main Thread] WARNING: NS_ENSURE_SUCCESS(rv, rv) failed with result 0x80520001: file /builds/worker/workspace/build/src/modules/libpref/Preferences.cpp, line 4503
[task 2019-11-22T13:28:16.396Z] 13:28:16     INFO -  caps/tests/unit/test_origin.js | [2957, Main Thread] WARNING: NS_ENSURE_TRUE(inst) failed: file StaticComponents.cpp, line 7283
[task 2019-11-22T13:28:16.396Z] 13:28:16     INFO -  caps/tests/unit/test_origin.js | [2957, Main Thread] WARNING: NS_ENSURE_TRUE(InitStaticMembers()) failed: file /builds/worker/workspace/build/src/modules/libpref/Preferences.cpp, line 4992
[task 2019-11-22T13:28:16.396Z] 13:28:16     INFO -  caps/tests/unit/test_origin.js | Segmentation fault
[task 2019-11-22T13:28:16.396Z] 13:28:16     INFO -  <<<<<<<

https://firefoxci.taskcluster-artifacts.net/MWMxKLWXQ4adwOYFCiwh8A/0/public/test_info//logcat-emulator-5554.log

11-22 13:28:15.385  2957  2957 I Gecko   : [2957, Unnamed thread 79014ed21020] WARNING: XPCOM objects created/destroyed from static ctor/dtor: file /builds/worker/workspace/build/src/xpcom/base/nsTraceRefcnt.cpp, line 198
11-22 13:28:15.385  2957  2957 I Gecko   : [2957, Unnamed thread 79014ed21020] WARNING: XPCOM objects created/destroyed from static ctor/dtor: file /builds/worker/workspace/build/src/xpcom/base/nsTraceRefcnt.cpp, line 198
11-22 13:28:15.385  2957  2957 I Gecko   : [2957, Unnamed thread 79014ed21020] WARNING: XPCOM objects created/destroyed from static ctor/dtor: file /builds/worker/workspace/build/src/xpcom/base/nsTraceRefcnt.cpp, line 198
11-22 13:28:15.386  2957  2957 I Gecko   : [2957, Unnamed thread 79014ed21020] WARNING: XPCOM objects created/destroyed from static ctor/dtor: file /builds/worker/workspace/build/src/xpcom/base/nsTraceRefcnt.cpp, line 198
11-22 13:28:15.387  2957  2957 I Gecko   : [2957, Main Thread] WARNING: No Android crash handler set: file /builds/worker/workspace/build/src/toolkit/crashreporter/nsExceptionHandler.cpp, line 1549
11-22 13:28:15.428  2957  2957 I Gecko   : [2957, Main Thread] WARNING: NS_ENSURE_SUCCESS(rv, rv) failed with result 0x80520001: file /builds/worker/workspace/build/src/modules/libpref/Preferences.cpp, line 4503
11-22 13:28:15.428  2957  2957 I Gecko   : [2957, Main Thread] WARNING: NS_ENSURE_TRUE(inst) failed: file StaticComponents.cpp, line 7283
11-22 13:28:15.428  2957  2957 I Gecko   : [2957, Main Thread] WARNING: NS_ENSURE_TRUE(InitStaticMembers()) failed: file /builds/worker/workspace/build/src/modules/libpref/Preferences.cpp, line 4992
11-22 13:28:15.428  2957  2957 F MOZ_Assert: Assertion failure: !gAccessCounts, at /builds/worker/workspace/build/src/modules/libpref/Preferences.cpp:3418

11-22 13:28:15.428  2957  2957 I Gecko   : [2957, Main Thread] WARNING: NS_ENSURE_SUCCESS(rv, rv) failed with result 0x80520001: file /builds/worker/workspace/build/src/modules/libpref/Preferences.cpp, line 4503

https://searchfox.org/mozilla-central/rev/3483fb259b4edbe4594cfcc3911db97d5441b67d/modules/libpref/Preferences.cpp#4503

That seems odd. Why would greprefs.js not be found? Wrong abi?

(In reply to Geoff Brown [:gbrown] from comment #2)

That seems odd. Why would greprefs.js not be found? Wrong abi?

Yes! I caught this in a debug try run, which attempted to retrieve the abi property a second time in case the first attempt failed intermittently:

https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=277769911&repo=try&lineNumber=8144

[task 2019-11-23T02:30:46.374Z] 02:30:46     INFO -  adb shell_output: adb -s emulator-5554 wait-for-device shell getprop ro.product.cpu.abi; echo adb_returncode=$?, timeout: None, root: False, timedout: None, exitcode: None, output:
...
[task 2019-11-23T02:30:46.889Z] 02:30:46     INFO -  adb shell_output: adb -s emulator-5554 wait-for-device shell getprop ro.product.cpu.abi; echo adb_returncode=$?, timeout: None, root: False, timedout: None, exitcode: 0, output: x86_64
[task 2019-11-23T02:30:46.889Z] 02:30:46     INFO -  x86_64
...
[task 2019-11-23T02:30:46.994Z] 02:30:46     INFO -  {'MOZ_CRASHREPORTER': '1', 'MOZ_WEBRENDER': '0', 'XPCOM_DEBUG_BREAK': 'stack-and-abort', 'XPCSHELL_MINIDUMP_DIR': '/sdcard/tests/xpc/minidumps', 'MOZ_DISABLE_CONTENT_SANDBOX': '1', 'XPCSHELL_TEST_TEMP_DIR': '/sdcard/tests/xpc/tmp', 'MOZ_ANDROID_DATA_DIR': '/data/local/xpcb', 'GRE_HOME': '/data/data/org.mozilla.geckoview.test', 'MOZ_DISABLE_NONLOCAL_CONNECTIONS': '1', 'MOZ_ANDROID_CPU_ABI': '', 'XPCSHELL_TEST_PROFILE_DIR': '/sdcard/tests/xpc/p', 'HOME': '/sdcard/tests/xpc/p', 'MOZ_CRASHREPORTER_NO_REPORT': '1', 'MOZ_DEVELOPER_REPO_DIR': '/builds/worker/workspace/build/src', 'MOZ_LINKER_CACHE': '/data/local/xpcb', 'LD_LIBRARY_PATH': '/data/local/xpcb', 'TMPDIR': '/sdcard/tests/xpc/tmp'}

So 'getprop ro.product.cpu.abi' returned '', resulting in MOZ_ANDROID_CPU_ABI='', which affects the greprefs.js search in xpcshell:

https://searchfox.org/mozilla-central/rev/3483fb259b4edbe4594cfcc3911db97d5441b67d/modules/libpref/Preferences.cpp#4488

On retry, 'getprop ro.product.cpu.abi' correctly returned 'x86_64'.

When running xpcshell on the x86_64 emulator, there are infrequent intermittent
test failures ultimately caused by an incorrect (blank) value being returned
from 'adb shell getprop ro.product.cpu.abi'. Reproduction of the issue on try
suggests that a simple retry avoids the issue.

https://hg.mozilla.org/mozilla-central/rev/d7657fa4772a