Closed Bug 1088538 Opened 10 years ago Closed 10 years ago

[RTSP] System crash in the end of playback due to NULL accessUnit

Categories

(Firefox OS Graveyard :: RTSP, defect)

ARM
Gonk (Firefox OS)
defect
Not set
critical

Tracking

(blocking-b2g:2.0+, firefox34 wontfix, firefox35 wontfix, firefox36 fixed, b2g-v2.0 fixed, b2g-v2.0M fixed, b2g-v2.1 fixed, b2g-v2.1S fixed, b2g-v2.2 fixed)

RESOLVED FIXED
2.1 S8 (7Nov)
blocking-b2g 2.0+
Tracking Status
firefox34 --- wontfix
firefox35 --- wontfix
firefox36 --- fixed
b2g-v2.0 --- fixed
b2g-v2.0M --- fixed
b2g-v2.1 --- fixed
b2g-v2.1S --- fixed
b2g-v2.2 --- fixed

People

(Reporter: ethan, Assigned: jhao)

References

Details

(Keywords: verifyme)

Attachments

(2 files, 1 obsolete file)

When the RTSP playback reaches the end of stream, the system could crash at this line: http://dxr.mozilla.org/mozilla-central/source/netwerk/protocol/rtsp/rtsp/RTSPSource.cpp#716
Attached file gdb-backtrace.log (deleted) —
From the GDB backtrace log, it seems somehow the accessUnit becomes NULL in RTSPSource::onTrackDataAvailable(). The assertion MOZ_ASSERT(accessUnit != NULL) only works in debug build, so it doesn't catch this error.
Jonathan, Please help to see if you can reproduce this bug.
Flags: needinfo?(jhao)
I doubt this bug is a recent regression. Before the patch of bug 1045062, the assertion is CHECK(accessUnit != NULL), and it should cause a crash if the assertion fails. But we didn't encounter such crash previously. https://hg.mozilla.org/mozilla-central/diff/ad7a09962cc4/netwerk/protocol/rtsp/rtsp/RTSPSource.cpp
Yes, I can reproduce. I'll look into this.
Assignee: nobody → jhao
Flags: needinfo?(jhao)
Summary: [RTSP] System crash in the end of playback → [RTSP] System crash in the end of playback due to NULL accessUnit
Hi Howie, This bug is critical. It shall block 2.2.
blocking-b2g: --- → 2.2?
Flags: needinfo?(hochang)
Blocks: 1032111
[Triage] It's a regression bug and causes the system crash.
blocking-b2g: 2.2? → 2.2+
Attached patch fix-1088538.patch (obsolete) (deleted) — Splinter Review
Please try out this patch. It may work as a workaround if not solving the underlying problem. I'll explain later.
Flags: needinfo?(hochang)
The variable accessUnit should be assigned by this line: > status_t err = dequeueAccessUnit(info->mIsAudio, &accessUnit); Then some error handling follows: > if (err == -EWOULDBLOCK) { > return; > } else if (err == INFO_DISCONTINUITY) { > ... > return; > } However, I found that dequeueAccessUnit may produce error code that is not OK, -EWOULDBLOCK nor INFO_DISCONTINUITY, so I added a third case > } else if (err != OK) { > return; > } After discussing with Benjamin and Vincent offline, we looked deeper and found that the unexpected error was ERROR_END_OF_STREAM, which should be expected because it was the end of stream. Therefore, in this revised patch, I removed the third case and let the function return when err == ERROR_END_OF_STREAM. Benjamin said we should try to combine this with our current end-of-stream mechanism that uses timer.
Attachment #8511796 - Attachment is obsolete: true
Attachment #8511851 - Flags: review?(ettseng)
Jonathan, Thanks for identifying the cause. From your explanation, I can understand: RTSPSource::dequeueAccessUnit() invokes AnotherPacketSource::dequeueAccessUnit(), which could possibly return ERROR_END_OF_STREAM, which is defined in MediaErrors.h. http://androidxref.com/4.4_r1/xref/frameworks/av/include/media/stagefright/MediaErrors.h Thus, RTSPSource::onTrackAvailable() should deal with the error code ERROR_END_OF_STREAM. It looks reasonable and I'll grant the patch. But could we explain why this issue doesn't occur previously? It's better if we know what previous change caused this side effect. :)
Attachment #8511851 - Flags: review?(ettseng) → review+
The patch's been put on try server. https://treeherder.mozilla.org/ui/#/jobs?repo=try&revision=5a3244e29b9a I don't know why it didn't happen before. The ERROR_END_OF_STREAM comes from this method: http://androidxref.com/4.4.4_r1/xref/frameworks/av/media/libstagefright/mpeg2ts/AnotherPacketSource.cpp#256 Maybe some of our recent patches affect this or there's some change to the framework.
Keywords: checkin-needed
Jonathan, I just refreshed my memory about end-of-stream by reading my own comments several months ago. For your reference: https://bugzilla.mozilla.org/show_bug.cgi?id=1021006#c12 As discussed offline, although the patch avoids the crash, the access units remaining in the queue might cause problem for re-play after EOS. Maybe we could clean up the queue of access units in performPlaybackEnded(). Please file another bug to track this issue.
(In reply to Ethan Tseng [:ethan] from comment #11) > Jonathan, > > I just refreshed my memory about end-of-stream by reading my own comments > several months ago. > For your reference: > https://bugzilla.mozilla.org/show_bug.cgi?id=1021006#c12 > > As discussed offline, although the patch avoids the crash, the access units > remaining in the queue > might cause problem for re-play after EOS. The replay after EOS will be a seek. So maybe there is no more problem. > Maybe we could clean up the queue of access units in performPlaybackEnded(). > Please file another bug to track this issue. Agree, clear them at performPlaybackEnded() is a good idea.
(In reply to Ethan Tseng [:ethan] from comment #11) > Please file another bug to track this issue. Here it is: bug 1090838.
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
Target Milestone: --- → 2.1 S8 (7Nov)
Couldn't repro this manually, Back-end issue
QA Whiteboard: [QAnalyst-Triage?][QAnalyst-verify-]
Flags: needinfo?(ktucker)
QA Whiteboard: [QAnalyst-Triage?][QAnalyst-verify-] → [QAnalyst-Triage+][QAnalyst-verify-]
Flags: needinfo?(ktucker)
Comment on attachment 8511851 [details] [diff] [review] Let onTrackDataAvailable return when getting ERROR_END_OF_STREAM [Approval Request Comment] Bug caused by (feature/regressing bug #): Bug 1111978 User impact if declined: Potential system crash Testing completed: On v2.1 Risk to taking this patch (and alternatives if risky): None String or UUID changes made by this patch: None
Attachment #8511851 - Flags: approval-mozilla-b2g34?
Hi Bhavana, This patch prevents a terrible potential system crash from RTSP streaming. We encountered this kind of crash while working on bug 1111978. Could you help to grant this uplift request?
Flags: needinfo?(bbajaj)
blocking-b2g: 2.2+ → 2.1+
Flags: needinfo?(bbajaj)
Comment on attachment 8511851 [details] [diff] [review] Let onTrackDataAvailable return when getting ERROR_END_OF_STREAM Approving given the patch is low risk and avoids a serious crash. Requesting QA verification once this land on 2.1.
Attachment #8511851 - Flags: approval-mozilla-b2g34? → approval-mozilla-b2g34+
Keywords: verifyme
Comment on attachment 8511851 [details] [diff] [review] Let onTrackDataAvailable return when getting ERROR_END_OF_STREAM [Approval Request Comment] Bug caused by (feature/regressing bug #): Bug 1146251 User impact if declined: Possible system crash (device reboot) Testing completed: On mozilla-b2g32_v2_0 Risk to taking this patch (and alternatives if risky): None String or UUID changes made by this patch: None Bhavana, The project Midori is based on B2G 2.0, which does not include this patch and has potential crash issue while playing RTSP streaming. The change and risk of this patch is pretty low, could we land it to 2.0?
Flags: needinfo?(bbajaj)
Attachment #8511851 - Flags: approval-mozilla-b2g32?
blocking-b2g: 2.1+ → 2.0+
Flags: needinfo?(bbajaj)
Attachment #8511851 - Flags: approval-mozilla-b2g32? → approval-mozilla-b2g32+
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: