Snippet from try log https://treeherder.allizom.org/logviewer.html#?job_id=12501318&repo=try =================== 08:32:46 INFO - MEMORY STAT | vsize 208MB | residentFast 55MB | heapAllocated 18MB 08:32:48 INFO - ############ ErrorPage.js 08:32:48 INFO - JavaScript error: chrome://specialpowers/content/SpecialPowersObserver.js, line 110: NS_ERROR_NOT_INITIALIZED: Component returned failure code: 0xc1f30001 (NS_ERROR_NOT_INITIALIZED) [nsIMessageSender.sendAsyncMessage] 08:32:49 INFO - [Parent 1614] WARNING: waitpid failed pid:1714 errno:10: file /home/worker/workspace/gecko/ipc/chromium/src/base/process_util_posix.cc, line 267 08:32:49 INFO - -*- SettingsManager: Received: inner-window-destroyed for valid innerWindowID=2, cleanup. 08:32:49 INFO - 1444120364627 Marionette INFO Closed connection conn1 08:32:49 INFO - AlarmService: observe(): profile-change-teardown 08:32:49 INFO - AlarmService: uninit() 08:32:50 INFO - JavaScript error: chrome://specialpowers/content/SpecialPowersObserver.js, line 110: NS_ERROR_NOT_INITIALIZED: Component returned failure code: 0xc1f30001 (NS_ERROR_NOT_INITIALIZED) [nsIMessageSender.sendAsyncMessage] 08:32:51 INFO - [Parent 1614] WARNING: waitpid failed pid:1624 errno:10: file /home/worker/workspace/gecko/ipc/chromium/src/base/process_util_posix.cc, line 267 08:32:51 INFO - JavaScript error: chrome://specialpowers/content/SpecialPowersObserver.js, line 154: NS_ERROR_ILLEGAL_VALUE: Component returned failure code: 0x80070057 (NS_ERROR_ILLEGAL_VALUE) [nsIObserverService.removeObserver] 08:32:51 INFO - -*- TCPPresentationServer.js: TCPPresentationServer - close 08:32:51 INFO - JavaScript error: jar:file:///system/b2g/omni.ja!/components/NfcContentHelper.js, line 257: TypeError: setting a property that has only a getter 08:32:52 INFO - JavaScript error: resource://gre/modules/IndexedDBHelper.jsm, line 121: UnknownError: The operation failed for reasons unrelated to the database itself and not covered by any other error code. 08:32:54 INFO - Segmentation fault 08:33:00 INFO - mozcrash Downloading symbols from: https://queue.taskcluster.net/v1/task/VP27_spKTpyTFMui46sD-w/artifacts/public/build/emulator.crashreporter-symbols.zip 08:33:00 INFO - Traceback (most recent call last): 08:33:00 INFO - File "/home/worker/build/venv/local/lib/python2.7/site-packages/mozrunner/base/runner.py", line 200, in check_for_crashes 08:33:00 INFO - test=test_name) 08:33:00 INFO - File "/home/worker/build/venv/local/lib/python2.7/site-packages/mozcrash/mozcrash.py", line 123, in log_crashes 08:33:00 INFO - stackwalk_binary=stackwalk_binary): 08:33:00 INFO - File "/home/worker/build/venv/local/lib/python2.7/site-packages/mozcrash/mozcrash.py", line 202, in __iter__ 08:33:00 INFO - rv = self._process_dump_file(path, extra) 08:33:00 INFO - File "/home/worker/build/venv/local/lib/python2.7/site-packages/mozcrash/mozcrash.py", line 224, in _process_dump_file 08:33:00 INFO - self._get_symbols() 08:33:00 INFO - File "/home/worker/build/venv/local/lib/python2.7/site-packages/mozcrash/mozcrash.py", line 171, in _get_symbols 08:33:00 INFO - data = urllib2.urlopen(self.symbols_path) 08:33:00 INFO - File "/usr/lib/python2.7/urllib2.py", line 126, in urlopen 08:33:00 INFO - return _opener.open(url, data, timeout) 08:33:00 INFO - File "/usr/lib/python2.7/urllib2.py", line 406, in open 08:33:00 INFO - response = meth(req, response) 08:33:00 INFO - File "/usr/lib/python2.7/urllib2.py", line 519, in http_response 08:33:00 INFO - 'http', request, response, code, msg, hdrs) 08:33:00 INFO - File "/usr/lib/python2.7/urllib2.py", line 444, in error 08:33:00 INFO - return self._call_chain(*args) 08:33:00 INFO - File "/usr/lib/python2.7/urllib2.py", line 378, in _call_chain 08:33:00 INFO - result = func(*args) 08:33:00 INFO - File "/usr/lib/python2.7/urllib2.py", line 527, in http_error_default 08:33:00 INFO - raise HTTPError(req.get_full_url(), code, msg, hdrs, fp) 08:33:01 INFO - HTTPError: HTTP Error 404: Not Found

I saw same error on xpcshell as well [1]. I check the log, it turns out mozcrash use a wrong url to download the symbols from taskcluster. The file name should be b2g.crashreporter-symbols.zip [2]. > 09:25:55 INFO - mozcrash Downloading symbols from: https://queue.taskcluster.net/v1/task/Q9XXlVEDRfa3_wyFw9ConQ/artifacts/public/build/emulator.crashreporter-symbols.zip [1] https://treeherder.mozilla.org/logviewer.html#?job_id=14606336&repo=try [2] https://tools.taskcluster.net/task-inspector/#Q9XXlVEDRfa3_wyFw9ConQ/0

We have defined the symbol location in task definition in taskcluster [1], but seems not pass the symbols_url to mozharness [2]. And TestingMixin only read the symbols_url from buildbot configuration [3], looks like it needs to add --symbols_url option in order to get symbols_url passed from taskcluster. [1] https://dxr.mozilla.org/mozilla-central/source/testing/taskcluster/tasks/builds/b2g_emulator_base.yml?from=b2g_emulator_base.yml#36 [2] https://dxr.mozilla.org/mozilla-central/source/testing/taskcluster/tasks/tests/b2g_emulator_xpcshell.yml#15-23 [3] https://dxr.mozilla.org/mozilla-central/source/testing/mozharness/mozharness/mozilla/testing/testbase.py#255-290

All mochiest tests become orange with having correct symbols_url [1]. And I saw a crash on mochiest shutdown (probably same as bug 1208418). -- 12:43:12 WARNING - PROCESS-CRASH | dom/apps/tests/test_bug_795164.html | application crashed [None] 12:43:12 INFO - Crash dump filename: /tmp/tmpwzDNfy/28802bb4-1503-e040-3061d4e4-3be2fa96.dmp 12:43:12 INFO - MINIDUMP_STACKWALK not set, can't process dump. -- [1] https://treeherder.mozilla.org/#/jobs?repo=try&revision=54b9e7d32c60&exclusion_profile=false&group_state=expanded&selectedJob=14648070

(In reply to Edgar Chen [:edgar][:echen] from comment #4) > All mochiest tests become orange with having correct symbols_url [1]. And I > saw a crash on mochiest shutdown (probably same as bug 1208418). > > -- > 12:43:12 WARNING - PROCESS-CRASH | dom/apps/tests/test_bug_795164.html | > application crashed [None] > 12:43:12 INFO - Crash dump filename: > /tmp/tmpwzDNfy/28802bb4-1503-e040-3061d4e4-3be2fa96.dmp > 12:43:12 INFO - MINIDUMP_STACKWALK not set, can't process dump. Let me try to make minidump works to see if we can get more useful information. > -- > > [1] > https://treeherder.mozilla.org/#/ > jobs?repo=try&revision=54b9e7d32c60&exclusion_profile=false&group_state=expan > ded&selectedJob=14648070

Looks like we have to update docker image to make minidump work.

Review commit: https://reviewboard.mozilla.org/r/29039/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/29039/

Comment on attachment 8701844 [details] MozReview Request: Bug 1215084 - Part 1: Add symbols-url option to emulator tests for taskcluster Review request updated; see interdiff: https://reviewboard.mozilla.org/r/29039/diff/1-2/

Review commit: https://reviewboard.mozilla.org/r/29365/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/29365/

Review commit: https://reviewboard.mozilla.org/r/29367/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/29367/

Comment on attachment 8701844 [details] MozReview Request: Bug 1215084 - Part 1: Add symbols-url option to emulator tests for taskcluster https://reviewboard.mozilla.org/r/29039/#review26163 It would be nice if we could fix https://dxr.mozilla.org/mozilla-central/source/testing/mozharness/mozharness/mozilla/testing/testbase.py#159 to be taskcluster aware. We'll need to get rid of buildbot specific stuff like this in mozharness anyway. But this patch looks reasonable either way, so feel free to land it. Would be good to at least file a follow-up bug though. Thanks for also digging into this!

Comment on attachment 8703482 [details] MozReview Request: Bug 1215084 - Part 3: Set MINIDUMP_STACKWALK to tester image https://reviewboard.mozilla.org/r/29367/#review26349

Comment on attachment 8703481 [details] MozReview Request: Bug 1215084 - Part 2: Pass correct symbols url to emulator tests https://reviewboard.mozilla.org/r/29365/#review26351

Hi Wander, I don't have permission to push image to taskcluster, could you help me to update the docker image or do you know what process I need to do to get the permission? Thank you.

(In reply to Andrew Halberstadt [:ahal] from comment #11) > Comment on attachment 8701844 [details] > MozReview Request: Bug 1215084 - Part 1: Add symbols-url option to emulator > tests for taskcluster > > https://reviewboard.mozilla.org/r/29039/#review26163 > > It would be nice if we could fix > https://dxr.mozilla.org/mozilla-central/source/testing/mozharness/mozharness/ > mozilla/testing/testbase.py#159 to be taskcluster aware. We'll need to get > rid of buildbot specific stuff like this in mozharness anyway. > > But this patch looks reasonable either way, so feel free to land it. Would > be good to at least file a follow-up bug though. Thanks for also digging > into this! File bug 1237182 as a follow-up bug.

(In reply to Edgar Chen [:edgar][:echen] from comment #14) > Hi Wander, I don't have permission to push image to taskcluster, could you > help me to update the docker image or do you know what process I need to do > to get the permission? Thank you. Do you have an account in docker hub?

(In reply to Wander Lairson Costa [:wcosta] from comment #16) > (In reply to Edgar Chen [:edgar][:echen] from comment #14) > > Hi Wander, I don't have permission to push image to taskcluster, could you > > help me to update the docker image or do you know what process I need to do > > to get the permission? Thank you. > > Do you have an account in docker hub? Yes, I have. My account is edgarchen, https://hub.docker.com/u/edgarchen/.

(In reply to Edgar Chen [:edgar][:echen] from comment #17) > (In reply to Wander Lairson Costa [:wcosta] from comment #16) > > (In reply to Edgar Chen [:edgar][:echen] from comment #14) > > > Hi Wander, I don't have permission to push image to taskcluster, could you > > > help me to update the docker image or do you know what process I need to do > > > to get the permission? Thank you. > > > > Do you have an account in docker hub? > > Yes, I have. My account is edgarchen, https://hub.docker.com/u/edgarchen/. I added you to the firefoxosdev group, please check if you can push the image now

(In reply to Wander Lairson Costa [:wcosta] from comment #18) > I added you to the firefoxosdev group, please check if you can push the > image now I can push the image now. Thank you, Wander.

Try result: https://treeherder.allizom.org/#/jobs?repo=try&revision=4d73e2b4387e&group_state=expanded&exclusion_profile=false

https://hg.mozilla.org/integration/b2g-inbound/rev/4d4178e60cb9 https://hg.mozilla.org/integration/b2g-inbound/rev/9cba1315b62e https://hg.mozilla.org/integration/b2g-inbound/rev/22b61066991a

https://hg.mozilla.org/mozilla-central/rev/4d4178e60cb9 https://hg.mozilla.org/mozilla-central/rev/9cba1315b62e https://hg.mozilla.org/mozilla-central/rev/22b61066991a