Inder Reporter
	Description • 10 years ago

STR:
1. Start camera app
2. Switch to camcorder
3. Record a video
4. Stop the recording
Camera app hangs or crashes.
This is quite easily reproducible in 2-3 tries.
This issue does not reproduce on KK based devices.

	Inder Reporter
	Comment 5 • 10 years ago

Sotaro - can you please take a quick look at the attachments to see if you have any clues?

	Sotaro Ikeda [:sotaro]
	Comment 6 • 10 years ago

The following log seems to related to the crash. "1515870810" is "5A5A5A5A" in HEX. It say that the accessed memory is already freed. info->mStatus is OMXCodec's internal member. From it, the crash might be caused by OMXCodec's problem.

> 01-01 23:17:22.943  1243  1663 F OMXCodec: frameworks/av/media/libstagefright/OMXCodec.cpp:4420 CHECK_EQ( (int)info->mStatus,(int)OWNED_BY_US) failed: 1515870810 vs. 0

	Mike Lee [:mlee]
	Comment 7 • 10 years ago

(In reply to Sotaro Ikeda [:sotaro] from comment #6)
> The following log seems to related to the crash. "1515870810" is "5A5A5A5A"
> in HEX. It say that the accessed memory is already freed. info->mStatus is
> OMXCodec's internal member. From it, the crash might be caused by OMXCodec's
> problem.
> 
> > 01-01 23:17:22.943  1243  1663 F OMXCodec: frameworks/av/media/libstagefright/OMXCodec.cpp:4420 CHECK_EQ( (int)info->mStatus,(int)OWNED_BY_US) failed: 1515870810 vs. 0


Sotaro,

Are you saying this is an issue in CAF's code?

Thanks,
Mike

	Sotaro Ikeda [:sotaro]
	Comment 8 • 10 years ago

(In reply to Mike Lee [:mlee] from comment #7)
> (In reply to Sotaro Ikeda [:sotaro] from comment #6)
> > The following log seems to related to the crash. "1515870810" is "5A5A5A5A"
> > in HEX. It say that the accessed memory is already freed. info->mStatus is
> > OMXCodec's internal member. From it, the crash might be caused by OMXCodec's
> > problem.
> > 
> > > 01-01 23:17:22.943  1243  1663 F OMXCodec: frameworks/av/media/libstagefright/OMXCodec.cpp:4420 CHECK_EQ( (int)info->mStatus,(int)OWNED_BY_US) failed: 1515870810 vs. 0
> 
> 
> Sotaro,
> 
> Are you saying this is an issue in CAF's code?

It is just one possibility. At this time, it seems more possible than other possibilities.

	Hema Koka [:hema]
	Comment 9 • 10 years ago

NI back to Inder based on Sotaro's input on comment 8

	Inder Reporter
	Comment 10 • 10 years ago

I have tried multiple times to reproduce this issue in Android with the same code base and we don't see this issue there.
The crash does not reproduce on KK based devices with v2.2 either so essentially points to L related integration issue in FirefoxOS.

To be fair, it's possible the issue is in OMXCodec and getting exposed with FirefoxOS but we would appreciate help to figure it out.

What's interesting is that the crash is manifesting with different call stack every time.

	Inder Reporter
	Comment 12 • 10 years ago

Here is another one.

	Sotaro Ikeda [:sotaro]
	Comment 13 • 10 years ago

(In reply to Inder from comment #10)
> I have tried multiple times to reproduce this issue in Android with the same
> code base and we don't see this issue there.
> The crash does not reproduce on KK based devices with v2.2 either so
> essentially points to L related integration issue in FirefoxOS.

On android, when we do recording, hw codec instance and MPEG4Writer is in same process. But in b2g, the hw codec instance is in mediaserver, but the MPEG4Writer is in a b2g-content process. That might make a difference.

 
> To be fair, it's possible the issue is in OMXCodec and getting exposed with
> FirefoxOS but we would appreciate help to figure it out.
> 
> What's interesting is that the crash is manifesting with different call
> stack every time.

The crash call stack seems not work to fix this kind of problem. We might need to track all function call sequence of OMXCodec and related objects. mozilla side still do not have a environment to do debugging :-( I received the device, but still could not flash Firefox OS to it...

	Sotaro Ikeda [:sotaro]
	Comment 14 • 10 years ago

(In reply to Sotaro Ikeda [:sotaro] from comment #13)
> 
> I received the device, but still could not flash Firefox OS to it...

Taiwan office is closed until 2/23 because of chinese new year. After that they are going to work to address the ROM write problem with a partner.

	Hema Koka [:hema]
	Comment 15 • 10 years ago

(In reply to Sotaro Ikeda [:sotaro] from comment #13)
  
> > To be fair, it's possible the issue is in OMXCodec and getting exposed with
> > FirefoxOS but we would appreciate help to figure it out.
> > 
> > What's interesting is that the crash is manifesting with different call
> > stack every time.
> 
> The crash call stack seems not work to fix this kind of problem. We might
> need to track all function call sequence of OMXCodec and related objects.
> mozilla side still do not have a environment to do debugging :-( I received
> the device, but still could not flash Firefox OS to it...

Inder, 

Could you or someone from your team try to step through the functions and debug? I spoke to Sotaro offline and since the crash logs show nothing about the root-cause and we don't have the right working device, it becomes hard to debug this issue by weeding through the logs and stacktrace. He was saying that just enabling OMXCodec debug log may not provide enough info to analyze this problem. You may need to track all related functions' of OMXCodec enter and exit and see what is going on. At this point (since Moz doesn't have a working hardware yet), it looks like you guys are best equipped to dig in and figure out the problem. So, please try Sotaro's suggestion. 

Thanks
Hema

	Inder Reporter
	Comment 16 • 10 years ago

We debugged with our video team on this and it turns out the issue is because we are using OMXCodec for camcorder recording. MediaRecorder on Android L uses MediaCodec for encoding audio and video instead of OMXCodec so we need to switch gecko to use MediaCodec instead.

We see that Bug 1033935 already tries to enable MediaCodec for v2.2 and it has all the approvals too. Not sure what it's waiting for.
Also, based on https://bugzilla.mozilla.org/show_bug.cgi?id=1033935#c4 we need all those 4 bug fixes uplifted to v2.2.

	Hema Koka [:hema]
	Comment 17 • 10 years ago

Adding dependency bug based on previous comments from Inder

NI Bhavana/Mike Lee to follow up on uplift status on bugs that are required by CAF: https://bugzilla.mozilla.org/show_bug.cgi?id=1033935#c4

	Sotaro Ikeda [:sotaro]
	Comment 18 • 10 years ago

(In reply to Inder from comment #16)
> 
> We see that Bug 1033935 already tries to enable MediaCodec for v2.2 and it
> has all the approvals too. Not sure what it's waiting for.
> Also, based on https://bugzilla.mozilla.org/show_bug.cgi?id=1033935#c4 we
> need all those 4 bug fixes uplifted to v2.2.

Bug 1033935 is a bug for playback not for recording.

	Sotaro Ikeda [:sotaro]
	Comment 19 • 10 years ago

(In reply to Inder from comment #16)
> We debugged with our video team on this and it turns out the issue is
> because we are using OMXCodec for camcorder recording. MediaRecorder on
> Android L uses MediaCodec for encoding audio and video instead of OMXCodec
> so we need to switch gecko to use MediaCodec instead.
> 
> We see that Bug 1033935 already tries to enable MediaCodec for v2.2 and it
> has all the approvals too. Not sure what it's waiting for.
> Also, based on https://bugzilla.mozilla.org/show_bug.cgi?id=1033935#c4 we
> need all those 4 bug fixes uplifted to v2.2.

Camera recording via mozCamera api uses OMXCodec for a long time because how to recording is almost same to android. And the code also re-uses android's source code.

	Sotaro Ikeda [:sotaro]
	Comment 20 • 10 years ago

(In reply to Sotaro Ikeda [:sotaro] from comment #19)
> 
> Camera recording via mozCamera api uses OMXCodec for a long time because how
> to recording is almost same to android. And the code also re-uses android's
> source code.

These codes were just borrowed from android aosp code. Therefore, if we want to record via mozCamera api, it seems easier just to get android AOSP Lollipop as well.

	Sotaro Ikeda [:sotaro]
	Comment 21 • 10 years ago

From comment 18 and comment 20, the following fix is not related to recording problem.

> Also, based on https://bugzilla.mozilla.org/show_bug.cgi?id=1033935#c4 we
> need all those 4 bug fixes uplifted to v2.2.

	Sotaro Ikeda [:sotaro]
	Comment 22 • 10 years ago

In current gecko, GonkCameraSource and GonkRecorder are the classes that were borrwed from AOSP.

https://dxr.mozilla.org/mozilla-central/source/dom/camera/GonkRecorder.cpp
https://dxr.mozilla.org/mozilla-central/source/dom/camera/GonkCameraSource.cpp


From L AOSP, we might need to borrow StagefrightRecorder, CameraSource and SurfaceMediaSource classes.

http://androidxref.com/5.0.0_r2/xref/frameworks/av/media/libmediaplayerservice/StagefrightRecorder.cpp
http://androidxref.com/5.0.0_r2/xref/frameworks/av/media/libstagefright/CameraSource.cpp
http://androidxref.com/5.0.0_r2/xref/frameworks/av/media/libstagefright/MediaCodecSource.cpp

	Sotaro Ikeda [:sotaro]
	Comment 23 • 10 years ago

About camera recording, it might better to get input from the people who works for b2g porting to L.

	Sotaro Ikeda [:sotaro]
	Comment 24 • 10 years ago

Vincent Liu, John Lin, can you comment to comment 23?

	Vincent Liu[:vliu]
	Comment 25 • 10 years ago

When I was porting Android L on Nexus 5, I also got similar issues that the camera got hang after camera recording. 

1. When stoping the recording, video track would block at pthread_join(). You can read the below message for detailed information.
   https://bugzilla.mozilla.org/show_bug.cgi?id=1114477#c13

2. "invalid color converter" message pops up when generating thumbnail. You may try the blow WIP to make sure it helps for this bug.
   https://bugzilla.mozilla.org/show_bug.cgi?id=1120780#c3

	Sotaro Ikeda [:sotaro]
	Comment 26 • 10 years ago

Then, about the crash, attachment 8543814 [details] in Bug 1114477 seems to address the problem.

	cafbot (PoC: ggrisco)
	Comment 27 • 10 years ago

Observed on: 

Device: msm8909
Gonk Version: AU_LINUX_GECKO_LF.BR.1.2.3.00.00.00.000.083
Moz BuildID: 20150222002501
Manifest: https://www.codeaurora.org/cgit/quic/lf/b2g/manifest/tree/caf_AU_LINUX_GECKO_LF.BR.1.2.3.00.00.00.000.083.xml?h=release
Gecko Version: 37.0a2
Gaia:  http://git.mozilla.org/?p=releases/gaia.git;a=commit;h=be86a57c64529b651e67ba4ae87d70acffbc986a
Gecko: http://git.mozilla.org/?p=releases/gecko.git;a=commit;h=7aa23764fe2edd6d8e5116d237b93a05dc591751
Patches: bug 1067629, bug 1125940, bug 1125418, bug 1091307, bug 1132528, bug 1134821

	cafbot (PoC: ggrisco)
	Comment 30 • 10 years ago

Observed on: 

Device: msm8909
Gonk Version: AU_LINUX_GECKO_LF.BR.1.2.3.00.00.00.000.084
Moz BuildID: 20150223002503
Manifest: https://www.codeaurora.org/cgit/quic/lf/b2g/manifest/tree/caf_AU_LINUX_GECKO_LF.BR.1.2.3.00.00.00.000.084.xml?h=release
Gecko Version: 37.0a2
Gaia:  http://git.mozilla.org/?p=releases/gaia.git;a=commit;h=389542b71c89253c0d176d3b0bfb54e275c19bf1
Gecko: http://git.mozilla.org/?p=releases/gecko.git;a=commit;h=84e31b298b5e1eb249a07083f2c26dcc2a6798d2
Patches: bug 1067629, bug 1125940, bug 1125418, bug 1091307, bug 1132528, bug 1134821

	Inder Reporter
	Comment 33 • 10 years ago

What solution are we looking into for v2.2? Do we want to stick with OMXCodec with the change mentioned in comment 25 or updating the implementation as mentioned in comment 22 to switch to MediaCodec?
I am also following up with video folks to see if there are any side effects with changing android framework to keep using OMXCodec.

	cafbot (PoC: ggrisco)
	Comment 34 • 10 years ago

Observed on: 

Device: msm8909
Gonk Version: AU_LINUX_GECKO_LF.BR.1.2.3.00.00.00.000.085
Moz BuildID: 20150225002505
Manifest: https://www.codeaurora.org/cgit/quic/lf/b2g/manifest/tree/caf_AU_LINUX_GECKO_LF.BR.1.2.3.00.00.00.000.085.xml?h=release
Gecko Version: 37.0
Gaia:  http://git.mozilla.org/?p=releases/gaia.git;a=commit;h=ca64f2fe145909f31af266b1730874051ba76c78
Gecko: http://git.mozilla.org/?p=releases/gecko.git;a=commit;h=fe489912aac2f83b14e7286b8f92443f6c6b64a7
Patches: bug 1067629, bug 1132528, bug 1125418, bug 1091307, bug 1134821

	Sotaro Ikeda [:sotaro]
	Comment 37 • 10 years ago

(In reply to Inder from comment #33)
> What solution are we looking into for v2.2? Do we want to stick with
> OMXCodec with the change mentioned in comment 25 or updating the
> implementation as mentioned in comment 22 to switch to MediaCodec?
> I am also following up with video folks to see if there are any side effects
> with changing android framework to keep using OMXCodec.

If there is no problem/side effects of using OMXCodec, using OMXCodec seems better than MediaCodec.

	Sotaro Ikeda [:sotaro]
	Comment 38 • 10 years ago

(In reply to Sotaro Ikeda [:sotaro] from comment #37)
> 
> If there is no problem/side effects of using OMXCodec, using OMXCodec seems
> better than MediaCodec.

The code modification could become minimum and stable source code in b2g.

	cafbot (PoC: ggrisco)
	Comment 39 • 10 years ago

Observed on: 

Device: msm8909
Gonk Version: AU_LINUX_GECKO_LF.BR.1.2.3.00.00.00.000.086
Moz BuildID: 20150225002505
Manifest: https://www.codeaurora.org/cgit/quic/lf/b2g/manifest/tree/caf_AU_LINUX_GECKO_LF.BR.1.2.3.00.00.00.000.086.xml?h=release
Gecko Version: 37.0
Gaia:  http://git.mozilla.org/?p=releases/gaia.git;a=commit;h=ca64f2fe145909f31af266b1730874051ba76c78
Gecko: http://git.mozilla.org/?p=releases/gecko.git;a=commit;h=fe489912aac2f83b14e7286b8f92443f6c6b64a7
Patches: bug 1067629, bug 1132528, bug 1125418, bug 1091307

	Inder Reporter
	Comment 42 • 10 years ago

I have landed a patch for MPEG4Writer similar to comment 26 in our local builds to enable OMXCodec during  recording and it's there since AU 89+. Let's see if this issue reproduces again.

	Bobby Chien
	Comment 43 • 10 years ago

could you help on this?

	StevenLee[:slee]
	Comment 44 • 10 years ago

As comment 42, asking QA to verify whether the bug still exists.

	Parul Mathur [:pragmatic] - out of office - please ni? [:bajaj] instead
	Comment 45 • 10 years ago

Attempted to reproduce the steps in comment 0 of this bug on a Nexus 5 device flashed with the latest b2g v2.2 build. The Camera app hangs after creating a video, in the sense that all its buttons stop responding. If the Cards View is launched by long pressing the Home button and the Camera app is brought back as the main app, its buttons start responding again.

Steps to Reproduce
1. Start camera app
2. Switch to camcorder
3. Record a video
4. Stop the recording

Expected Results
Tapping on any other button works as expected.

Actual Results
Tapping on any other button does not produce any result, making it appear that the camera app is hung.

Steps to Reproduce (continued)
5. Switch to Cards View by long pressing the Software Home Button.
6. Switch back to the Camera app.

Expected Results
Tapping on any other button works as expected.

Actual Results
Tapping on any other button works as expected. At no point does the Camera app crash. Taking multiple photographs works as expected and produces no hangs or crashes. (See adb logcat attached as 1133147-logcat-20150311.txt)

Test Environment
Device-Name     hammerhead
Software        b2 2.2.0.0-prerelease 
Gaia-Rev        3f032238a52f08e4c2f68a47ad065a96eb22d470
Gecko-Rev       https://hg.mozilla.org/releases/mozilla-b2g37_v2_2/rev/004fa1cb1dd4
Build-ID        20150311002522
Version         37.0
FW-Release      5.0
FW-Incremental  eng.cltbld.20150311.040259
FW-Date         Wed Mar 11 04:03:14 EDT 2015
Bootloader      HHZ11n

	Pi Wei Cheng [:piwei] (inactive)
	Comment 46 • 10 years ago

Removing tag - See comment 45.

	Vincent Liu[:vliu]
	Comment 47 • 10 years ago

(In reply to Parul Mathur [:pragmatic] from comment #45)
> Created attachment 8576145 [details]
> 1133147-logcat-20150311.txt
> 
> Attempted to reproduce the steps in comment 0 of this bug on a Nexus 5
> device flashed with the latest b2g v2.2 build. The Camera app hangs after
> creating a video, in the sense that all its buttons stop responding. If the
> Cards View is launched by long pressing the Home button and the Camera app
> is brought back as the main app, its buttons start responding again.
> 
> Steps to Reproduce
> 1. Start camera app
> 2. Switch to camcorder
> 3. Record a video
> 4. Stop the recording
> 
> Expected Results
> Tapping on any other button works as expected.
> 
> Actual Results
> Tapping on any other button does not produce any result, making it appear
> that the camera app is hung.
> 
> Steps to Reproduce (continued)
> 5. Switch to Cards View by long pressing the Software Home Button.
> 6. Switch back to the Camera app.
> 
> Expected Results
> Tapping on any other button works as expected.
> 
> Actual Results
> Tapping on any other button works as expected. At no point does the Camera
> app crash. Taking multiple photographs works as expected and produces no
> hangs or crashes. (See adb logcat attached as 1133147-logcat-20150311.txt)
> 
> Test Environment
> Device-Name     hammerhead
> Software        b2 2.2.0.0-prerelease 
> Gaia-Rev        3f032238a52f08e4c2f68a47ad065a96eb22d470
> Gecko-Rev      
> https://hg.mozilla.org/releases/mozilla-b2g37_v2_2/rev/004fa1cb1dd4
> Build-ID        20150311002522
> Version         37.0
> FW-Release      5.0
> FW-Incremental  eng.cltbld.20150311.040259
> FW-Date         Wed Mar 11 04:03:14 EDT 2015
> Bootloader      HHZ11n

On current Nexus 5 for v2.2 branch, this hang is expected since the patch of bug 1120780 doesn't land into it.

	Bobby Chien
	Comment 48 • 10 years ago

(In reply to Vincent Liu[:vliu] from comment #47)
> (In reply to Parul Mathur [:pragmatic] from comment #45)
> > Created attachment 8576145 [details]
> > 1133147-logcat-20150311.txt
> > 
> > Attempted to reproduce the steps in comment 0 of this bug on a Nexus 5
> > device flashed with the latest b2g v2.2 build. The Camera app hangs after
> > creating a video, in the sense that all its buttons stop responding. If the
> > Cards View is launched by long pressing the Home button and the Camera app
> > is brought back as the main app, its buttons start responding again.
> > 
> > Steps to Reproduce
> > 1. Start camera app
> > 2. Switch to camcorder
> > 3. Record a video
> > 4. Stop the recording
> > 
> > Expected Results
> > Tapping on any other button works as expected.
> > 
> > Actual Results
> > Tapping on any other button does not produce any result, making it appear
> > that the camera app is hung.
> > 
> > Steps to Reproduce (continued)
> > 5. Switch to Cards View by long pressing the Software Home Button.
> > 6. Switch back to the Camera app.
> > 
> > Expected Results
> > Tapping on any other button works as expected.
> > 
> > Actual Results
> > Tapping on any other button works as expected. At no point does the Camera
> > app crash. Taking multiple photographs works as expected and produces no
> > hangs or crashes. (See adb logcat attached as 1133147-logcat-20150311.txt)
> > 
> > Test Environment
> > Device-Name     hammerhead
> > Software        b2 2.2.0.0-prerelease 
> > Gaia-Rev        3f032238a52f08e4c2f68a47ad065a96eb22d470
> > Gecko-Rev      
> > https://hg.mozilla.org/releases/mozilla-b2g37_v2_2/rev/004fa1cb1dd4
> > Build-ID        20150311002522
> > Version         37.0
> > FW-Release      5.0
> > FW-Incremental  eng.cltbld.20150311.040259
> > FW-Date         Wed Mar 11 04:03:14 EDT 2015
> > Bootloader      HHZ11n
> 
> On current Nexus 5 for v2.2 branch, this hang is expected since the patch of
> bug 1120780 doesn't land into it.