Closed Bug 990837 Opened 11 years ago Closed 10 years ago

[MTBF] Phone Hang/Crashed with no adb/fastboot

Categories

(Firefox OS Graveyard :: Gaia::System, defect, P3)

ARM
Gonk (Firefox OS)
defect

Tracking

(tracking-b2g:backlog)

RESOLVED WORKSFORME
tracking-b2g backlog

People

(Reporter: wachen, Unassigned)

References

Details

(Whiteboard: [systemsfe])

Attachments

(1 file)

Got 3/3 files. Pulled files into about-memory-153. To view this report, open Firefox on this machine and load the following URL: about:memory?file=/home/wachen/FirefoxOS/MTBF-Driver/about-memory-153/memory-reports Pulling GC/CC logs... Got 6/6 files. Pulled files into about-memory-153. Compressing logs... starting httpd running webserver on http://10.247.24.65:40837/ test_keyboard_basic (test_mtbf_keyboard.TestKeyboard) ... ok ---------------------------------------------------------------------- Ran 1 test in 56.519s OK test_number_keyboard (test_mtbf_number_keyboard.TestNumberKeyboard) ... ok ---------------------------------------------------------------------- Ran 1 test in 29.131s OK test_keyboard_predictive_key (test_mtbf_keyboard_predictive_key.TestKeyboardPredictiveKey) ... ok ---------------------------------------------------------------------- Ran 1 test in 38.276s OK test_basic_email_keyboard (test_mtbf_email_keyboard.TestEmailKeyboard) ... ERROR ====================================================================== ERROR: None ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/marionette_client-0.7.1-py2.7.egg/marionette/marionette_test.py", line 127, in run self.setUp() File "/home/wachen/FirefoxOS/gaia_for_uitest/tests/python/gaia-ui-tests/gaiatest/tests/mtbf/keyboard/test_mtbf_email_keyboard.py", line 14, in setUp GaiaMtbfTestCase.setUp(self) File "/home/wachen/FirefoxOS/MTBF-Driver/MtbfTestCase.py", line 74, in setUp self.cleanup_gaia(full_reset=True) File "/home/wachen/FirefoxOS/MTBF-Driver/MtbfTestCase.py", line 106, in cleanup_gaia for filename in self.data_layer.media_files: File "/home/wachen/FirefoxOS/gaia_for_uitest/tests/python/gaia-ui-tests/gaiatest/gaia_test.py", line 372, in media_files result.extend(self.music_files) File "/home/wachen/FirefoxOS/gaia_for_uitest/tests/python/gaia-ui-tests/gaiatest/gaia_test.py", line 392, in music_files 'return GaiaDataLayer.getAllMusic();') File "/usr/local/lib/python2.7/dist-packages/marionette_client-0.7.1-py2.7.egg/marionette/marionette.py", line 1073, in execute_async_script filename=os.path.basename(frame[0])) File "/usr/local/lib/python2.7/dist-packages/marionette_client-0.7.1-py2.7.egg/marionette/marionette.py", line 567, in _send_message raise TimeoutException(message='socket.timeout', status=ErrorCodes.TIMEOUT, stacktrace=None) TEST-UNEXPECTED-FAIL | test_mtbf_email_keyboard.py test_mtbf_email_keyboard.TestEmailKeyboard.test_basic_email_keyboard | TimeoutException: socket.timeout ---------------------------------------------------------------------- Ran 1 test in 363.185s FAILED (errors=1) test_url_keyboard (test_mtbf_url_keyboard.TestUrlKeyboard) ... ERROR ====================================================================== ERROR: None ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/marionette_client-0.7.1-py2.7.egg/marionette/marionette_test.py", line 127, in run self.setUp() File "/home/wachen/FirefoxOS/gaia_for_uitest/tests/python/gaia-ui-tests/gaiatest/tests/mtbf/keyboard/test_mtbf_url_keyboard.py", line 13, in setUp GaiaMtbfTestCase.setUp(self) File "/home/wachen/FirefoxOS/MTBF-Driver/MtbfTestCase.py", line 29, in setUp MarionetteTestCase.setUp(self) File "/usr/local/lib/python2.7/dist-packages/marionette_client-0.7.1-py2.7.egg/marionette/marionette_test.py", line 312, in setUp CommonTestCase.setUp(self) File "/usr/local/lib/python2.7/dist-packages/marionette_client-0.7.1-py2.7.egg/marionette/marionette_test.py", line 247, in setUp self.marionette.start_session() File "/usr/local/lib/python2.7/dist-packages/marionette_client-0.7.1-py2.7.egg/marionette/marionette.py", line 681, in start_session self.session = self._send_message('newSession', 'value') File "/usr/local/lib/python2.7/dist-packages/marionette_client-0.7.1-py2.7.egg/marionette/marionette.py", line 567, in _send_message raise TimeoutException(message='socket.timeout', status=ErrorCodes.TIMEOUT, stacktrace=None) TEST-UNEXPECTED-FAIL | test_mtbf_url_keyboard.py test_mtbf_url_keyboard.TestUrlKeyboard.test_url_keyboard | TimeoutException: socket.timeout ---------------------------------------------------------------------- Ran 1 test in 360.078s FAILED (errors=1) SUMMARY ------- passed: 3 failed: 2 todo: 0 FAILED TESTS ------- test_mtbf_email_keyboard.py test_mtbf_email_keyboard.TestEmailKeyboard.test_basic_email_keyboard test_mtbf_url_keyboard.py test_mtbf_url_keyboard.TestUrlKeyboard.test_url_keyboard
Blocks: MTBF-meta
Attached image Phone after ran the MTBF testing (deleted) —
no adb connection, no fastboot connection, lsusb works: Bus 003 Device 067: ID 05c6:9025 Qualcomm, Inc. I reproduced this in v1.2, v1.3, and v1.4. Simply use MTBF testing with keyboard test suite.
Memory Report Dropbox location: https://drive.google.com/file/d/0BwDTW9d_sBJMYTRpRjd2QXZ1ZEE/edit?usp=sharing about-memory-1: 40.62 MB (100.0%) -- explicit 92 (100.0%) -- event-counts about-memory-70: 67.33 MB (100.0%) -- explicit 6,925 (100.0%) -- event-counts about-memory-153: 92.76 MB (100.0%) -- explicit 14,395 (100.0%) -- event-counts This is increasing rapidly 13.90 MB (14.98%) -- compartment([System Principal], inProcessTabChildGlobal?ownedBy=chrome://browser/content/shell.html)
Hi, Robert, Can you help us once again? (If you don't have free time, please ignore this request) If possible, try to run keyboard testing for 12+ hours.
Flags: needinfo?(rwood)
Also, there is no crash report found after reboot.
Hi, Brian and Kevin, This can be duplicated in multiple branches, and it needs to be concerned. Please do help us on this on.
Flags: needinfo?(khu)
Flags: needinfo?(brhuang)
Ivan, Could you have someone to take a look on this? We use keyboard test scenarios to do test.
Flags: needinfo?(brhuang) → needinfo?(itsay)
I don't see the thing from comment 2 in the files in comment 5. I do see the "UI Test" process increasing a lot, and I see 9.47 MB (100.0%) -- explicit ├──3.92 MB (41.39%) ── freetype ├──3.87 MB (40.85%) -- workers │ ├──4.00 MB (42.23%) -- workers(gaiamobile.org)/worker(js/imes/latin/worker.js, 0xNNN) In the parent. I don't really know anything about IMEs, unfortunately.
Do we know what operations were done right before the crash? Thanks.
Flags: needinfo?(khu)
I'd say, we kind of know it's running basic keyboard testing. From the log, we can tell it is ok doing OK test_keyboard_predictive_key. That is to say, we were entering some string and see if predictive of words are correct. And, then it crashed in general cleaning up in setUp()
Hi, Kyle, 1. Result in comment 2 It's in memory-reports file. I was just pointing out the explicit memory of Main process going up from 40.62 MB -> 67.33 MB -> 92.76 MB. Also, event-counts from other measurement is going up from 92 to 6925 to 14395. 2. "UI Tests" process is not marionette or gaia-ui-tests, it's an app you can find in engineer build of any kind. You can open it up and get an overview of almost all UI. It's like Phone, Gallery, Camera, or any other kind of app. I saw it varies from 8MB to 15MB from time to time.
Flags: needinfo?(khuey)
Hi Gary, Mind if you can help QA for the trouble shooting on this case. So far not sure why this is happening. Need help to narrow down cause of the issue.
Flags: needinfo?(itsay) → needinfo?(gchen)
Oh. about-memory-0 from that folder seems to be from relatively late in the run ... which is why I was seeing small diffs. Compare about-memory-0 and about-memory-1 ... about-memory-134 also appears to have a complete GC/CC log pair for the b2g process. I'm investigating the issue now.
Flags: needinfo?(khuey)
So perusing the heap logs I notice that we have several hundred settings observers floating around. These come from apps/system/js/quick_settings.js. They're entraining requests which is what causes the inflated event-targets and event-counts numbers. Somehow it looks like quick_settings.js is being initialized multiple times. I wonder if http://mxr.mozilla.org/gaia/source/apps/system/js/quick_settings.js#341 should remove itself as an event listener. Is it possible to fire the "localized" event multiple times? Regardless, this is a gaia issue and needs to be owned by someone from that team.
Hi, Alive, can you help us to look into this issue?
Flags: needinfo?(alive)
Depends on: 992715
See comment I added in bug 992715
Flags: needinfo?(rwood)
It's not obvious what should be done here...I could make a patch to fix comment 15 anyway.
Flags: needinfo?(alive)
Flags: needinfo?(gchen)
(In reply to Alive Kuo [:alive][NEEDINFO!][God bless Taiwan.] from comment #18) > It's not obvious what should be done here...I could make a patch to fix > comment 15 anyway. Figuring out how this is actually happening would be a good start ...
(In reply to (Away 4/19-5/7) from comment #19) > (In reply to Alive Kuo [:alive][NEEDINFO!][God bless Taiwan.] from comment > #18) > > It's not obvious what should be done here...I could make a patch to fix > > comment 15 anyway. > > Figuring out how this is actually happening would be a good start ... +1 to this. Alive, if you need more information about the environment, can you check with Walter? And, do you think you're the good fit to own this bug? Thank you.
Flags: needinfo?(alive)
Blocked by blocker enough, flag Tim to find someone.
Flags: needinfo?(alive) → needinfo?(timdream)
Gregor, could you assign someone on your team for this script in the System app?
Component: Performance → Gaia::System
Flags: needinfo?(timdream) → needinfo?(anygregor)
Whiteboard: [systemfe]
You mean fixing quick_settings.js? I dont think anyone from my team has touched this file before. We can put it in our backlog but I am not sure when we get to it if it's not a blocker or a 2.0 feature.
Flags: needinfo?(anygregor)
Whiteboard: [systemfe] → [systemsfe]
blocking-b2g: --- → backlog
Blocks: MTBF-B2G
No longer blocks: MTBF-meta
Walter are you still seeing this issue with all the latest settings changes?
Flags: needinfo?(wachen)
Priority: -- → P3
We are currently in a totally different branch, and I am not seeing this on v2.1 currently because of the blocking of 1064800. I will need 1064800 to be solved first, and then I can look into it.
Haven't seem this for awhile
Status: NEW → RESOLVED
Closed: 10 years ago
Flags: needinfo?(wachen)
Resolution: --- → WORKSFORME
blocking-b2g: backlog → ---
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: