STR: Random stability testing for 24 hours Issue: device is stuck in camera app. I can take pic/ record video but I cannot go back to homescreen. Analysis: I can see both homescreen and camera app is running OOM_ADJ=2 (foreground app). Homescreen should not become foreground app when we are recording video or taking pic using camera app. I attached gc/cc logs when this is reproduced. @Kyle/@kevin: Could you please take a look at gc/cc logs. Device was displaying camera preview when this happened. FFOS: v2.1 Device : msm8926 RAM : 1GB Please note that we used following gaia/gecko and our build already has fix from bug 1055299 attachment 8487702 [details] (cherry-picked separately) https://www.codeaurora.org/cgit/quic/lf/b2g/mozilla/gaia/commit/?h=mozilla/v2.1&id=379e68fe729a684fa2fcddb30ea1e65508db73e1 https://www.codeaurora.org/cgit/quic/lf/b2g/mozilla/gecko/commit/?h=mozilla/v2.1&id=0696af755b2b86a34293761aa1164798d4b0be02

[Blocking Requested - why for this release]:

making NI to alive so that we get some idea what is happening in gaia window management

kyle/kgrandon : please look into gc/cc logs

Are you seeing this on 2.0 or 2.1? You nominated this for blocking 2.0, but everything else says 2.1.

[Blocking Requested - why for this release]:

(In reply to Kyle Huey [:khuey] (khuey@mozilla.com) from comment #4) > Are you seeing this on 2.0 or 2.1? You nominated this for blocking 2.0, but > everything else says 2.1. I am seeing it only in 2.1 . We are not seeing this issue in 2.0 anymore. Sorry for the confusion. Did you find anything from gc/cc logs.

The CC log only shows a single AppWindow div that is 'active'.

My first thought is that this may be related to bug 1067205, which could be putting the system into some weird state, but I have not done any investigation to validate this.

(In reply to Kyle Huey [:khuey] (khuey@mozilla.com) from comment #7) > The CC log only shows a single AppWindow div that is 'active'. But I confirmed from |b2g-info| log that both Homescreen and camera app has OOM_ADJ=2 for hours even if you keep device idle for a long time. Any idea what logging we should enable to debug this further ?

The process priority manager logging at http://mxr.mozilla.org/mozilla-central/source/dom/ipc/ProcessPriorityManager.cpp#46 would be a good start. Probably some window manager logging would be useful too.

Tapas - does this reproduce with any other app besides camera? If not, can you try with the patch from bug 1067205 to see if that fixes the issue? I can also provide a patch to enable more system logging (one that Alive has uploaded in other bugs before), but I would first like to understand the impact of bug 1067205.

(In reply to Kevin Grandon :kgrandon from comment #11) > Tapas - does this reproduce with any other app besides camera? I have seen it only once and camera app was running at that time. interesting point was that i can press record video/take pic using camera but if i press home key then homescreen is not displayed . Another interesting point was that both Homescreen and camera app is running with OOM_ADJ=2 when issue happened. > If not, can > you try with the patch from bug 1067205 to see if that fixes the issue? > I don't mind to try with that patch as it is going to land anyhow later. If it fixes this issue as well then its a bonus for us :) . > I can also provide a patch to enable more system logging (one that Alive has > uploaded in other bugs before), but I would first like to understand the > impact of bug 1067205. Can we enable both logging and fix from bug 1067205 ? This may cutoff debugging time. Please note that I want to enable this kind of logging using some |setprop| dynamically as it will help in future too. Basically, I want to cut down debugging effort for these kind of issues both for tester and developer.

hi Kevin, We also see this issue in v2.0 with video app playback but in v2.0, only video app was running with OOM_ADJ=2. May be a logging patch for both v2.0 and v2.1 can help here ? Please suggest. Please also let me know if I should apply fix from bug 1067205 in v2.0 too or not .

(In reply to Tapas[:tkundu on #b2g/gaia/memshrink/gfx] (always NI me) from comment #12) > Can we enable both logging and fix from bug 1067205 ? This may cutoff > debugging time. Please note that I want to enable this kind of logging using > some |setprop| dynamically as it will help in future too. Basically, I want > to cut down debugging effort for these kind of issues both for tester and > developer. I agree, we need to make logging easier to turn on in general and it's something we're working on. I don't know the best place for debugging, but I bet that flipping these flags would help: https://github.com/mozilla-b2g/gaia/blob/master/apps/system/js/app_window.js#L13 https://github.com/mozilla-b2g/gaia/blob/master/apps/system/js/app_window_manager.js#L20

Another monkey test bug? What's the exact behaviors to the device? Sounds really similar to https://bugzilla.mozilla.org/show_bug.cgi?id=1056216

So this sounds like it could also be a duplicate of bug 1055299?

We are also seeing this issue on v2.0 and we are not see multiple app running with OOM_ADJ=2 in v2.0 (confirmed from b2g-info log). But we have gc/cc logs and logcat logs with debug flag enabled from Comment 14. here is the complete log for v2.0 . https://drive.google.com/file/d/0B1cSMS8_GuAEUG1hcVB6U2ExWk0/edit?usp=sharing @kevin: Could you please take a look in v2.0 log and I also have a device ready for debugging in this state now. I am interested to know what is causing this error in logcat : 02-01 01:58:34.109 1005 1005 E GeckoConsole: Content JS ERROR at app://verticalhome.gaiamobile.org/gaia_build_defer_index.js:398 in GridItem.prototype.doRenderIcon/<: Error fetching icon [Exception... "File error: Not found" nsresult: "0x80520012 (NS_ERROR_FILE_NOT_FOUND)" location: "JS frame :: app://verticalhome.gaiamobile.org/gaia_build_defer_index.js :: fetchBlob/< :: line 380" data: no] Device is stuck with video app and i can play videos . But I cannot return to homescreen. If I try to return to homescreen then it prints following logs : http://pastebin.mozilla.org/6596586 This suggests that gaia is receiving home button but it is not able to show homescreen.

(In reply to Tapas[:tkundu on #b2g/gaia/memshrink/gfx] (always NI me) from comment #17) > @kevin: Could you please take a look in v2.0 log and I also have a device > ready for debugging in this state now. I am interested to know what is > causing this error in logcat : > > 02-01 01:58:34.109 1005 1005 E GeckoConsole: Content JS ERROR at > app://verticalhome.gaiamobile.org/gaia_build_defer_index.js:398 in > GridItem.prototype.doRenderIcon/<: Error fetching icon [Exception... "File > error: Not found" nsresult: "0x80520012 (NS_ERROR_FILE_NOT_FOUND)" > location: "JS frame :: > app://verticalhome.gaiamobile.org/gaia_build_defer_index.js :: fetchBlob/< > :: line 380" data: no] This should not be causing any problems, but it's something we should fix. I'm opening up a new bug to track this.

Looking through the logs, this is the most suspect error message to me: 02-01 16:09:43.049 237 237 E GeckoConsole: [JavaScript Error: "TypeError: this.element is null" {file: "app://system.gaiamobile.org/js/app_window.js" line: 1286}] 02-01 16:09:43.059 237 237 I Gecko : 2736583070 Marionette INFO sendToClient: {"from":"0","error":{"message":"TypeError: this.element is null","status":17,"stacktrace":"@app://system.gaiamobile.org/js/app_window.js, line 1286"}}, {2b4ef290-1856-46ad-82b4-44bd21bfd110}, null Alive - it looks like this is pointing to something in a resize handler: https://github.com/mozilla-b2g/gaia/blob/v2.0/apps/system/js/app_window.js#L1286 Do you think it's possible for this to be called when we don't have an app window element?

(In reply to Kevin Grandon :kgrandon from comment #19) > Looking through the logs, this is the most suspect error message to me: > > 02-01 16:09:43.049 237 237 E GeckoConsole: [JavaScript Error: > "TypeError: this.element is null" {file: > "app://system.gaiamobile.org/js/app_window.js" line: 1286}] > 02-01 16:09:43.059 237 237 I Gecko : 2736583070 Marionette INFO > sendToClient: {"from":"0","error":{"message":"TypeError: this.element is > null","status":17,"stacktrace":"@app://system.gaiamobile.org/js/app_window. > js, line 1286"}}, {2b4ef290-1856-46ad-82b4-44bd21bfd110}, null > > Alive - it looks like this is pointing to something in a resize handler: > https://github.com/mozilla-b2g/gaia/blob/v2.0/apps/system/js/app_window. > js#L1286 > > Do you think it's possible for this to be called when we don't have an app > window element? This looks like another variation of https://bugzilla.mozilla.org/show_bug.cgi?id=1031225 Possible because homescreen window element might be removed. I need to know the call path to the resize function. So Tapas, is there a stable STR? Could you repro every time now? Going to make a debugging patch.

(In reply to Alive Kuo [:alive][NEEDINFO!] from comment #20) > This looks like another variation of > https://bugzilla.mozilla.org/show_bug.cgi?id=1031225 > > Possible because homescreen window element might be removed. I need to know > the call path to the resize function. > > So Tapas, is there a stable STR? Could you repro every time now? Going to > make a debugging patch. STR is stable for me ONLY. It takes minimum 24 hours for me to reproduce it. Please give me a patch for both v2.0 and v2.1. Thanks a lot for quick help on this.

Patch for v2.0

Patch for v2.1

Tapas, any update here?

Sorry, I need to re-run same test again. I will update here after 24 hours of running test.

(In reply to Tapas[:tkundu on #b2g/gaia/memshrink/gfx] (always NI me) from comment #25) > Sorry, I need to re-run same test again. I will update here after 24 hours > of running test. Any news here?

still waiting for our test team to reproduce this again with your patch. Please expect delay for 3 more days.

Tapas, can we close this now?

(In reply to Gregor Wagner [:gwagner] from comment #28) > Tapas, can we close this now? Still waiting our test team response. Sorry for unexpected delays .

Lets re-open if it reproduces.

We are not hitting this issue anymore. We will reopen it if we see it again !