Treeherder Bug Filer Reporter
	Description • 7 years ago
treeherder

Filed by: bclary [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=124941342&repo=autoland

https://autophone.s3.amazonaws.com/v1/task/a4uZnterQ32e6RiVe1PnKg/runs/0/artifacts/public/build/00a0684f-a76c-436a-9aaf-2eca64989b52-autophone.log

	Alastor Wu [:alwu]
	Comment 1 • 7 years ago

From the log, I saw the binder died so we got the decoded error.

> E/GeckoRemoteManager( 8708): remote codec is dead

Hi, John,
Is it possibly a OOP issue?

	John Lin [:jhlin][:jolin] Assignee
	Comment 2 • 7 years ago

(In reply to Alastor Wu [:alwu][please needinfo me][GMT+8] from comment #1)
> Created attachment 8900062 [details]
> Error log
> 
> From the log, I saw the binder died so we got the decoded error.
> 
> > E/GeckoRemoteManager( 8708): remote codec is dead
> 
> Hi, John,
> Is it possibly a OOP issue?

 Yes, the remote decoder process is dead during sending input buffer. The error/exception handling code should be changed to return non fatal error (rather than OOM error) for reader to retry.

	John Lin [:jhlin][:jolin] Assignee
	Comment 8 • 7 years ago

Looks like a problem in media shutdown process:

11-20 15:30:25.872 I/Gecko   ( 6404): FATAL ERROR: AsyncShutdown timeout in profile-before-change Conditions: [{"name":"MediaShutdownManager: shutdown","state":"(none)","filename":"/builds/worker/workspace/build/src/dom/media/MediaShutdownManager.cpp","lineNumber":83,"stack":"MediaShutdownManager shutdown"}] At least one completion condition failed to complete within a reasonable amount of time. Causing a crash to ensure that we do not leave the user with an unresponsive process draining resources.

	JW Wang [:jwwang] [:jw_wang]
	Comment 13 • 7 years ago
mozreview-review

Comment on attachment 8933580 [details]
Bug 1392747 - add debug message to trace media shutdown process.

https://reviewboard.mozilla.org/r/204518/#review210092

	Pulsebot
	Comment 15 • 7 years ago

Pushed by jolin@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/6524c2d721db
add debug message to trace media shutdown process. r=jwwang

	Bogdan Tara[:bogdan_tara | bogdant]
	Comment 17 • 7 years ago
bugherder

https://hg.mozilla.org/mozilla-central/rev/6524c2d721db

	Ryan VanderMeulen [:RyanVM]
	Comment 18 • 7 years ago

Looks like this wasn't meant to be resolved yet.

	John Lin [:jhlin][:jolin] Assignee
	Comment 19 • 7 years ago

Thanks a lot, Ryan.

Added leave-open keyword until the root cause is found and fixed.

	Chris Peterson [:cpeterson]
	Comment 24 • 7 years ago

Jon, I see lots of these "DEBUG SHUTDOWN" log messages when running Nightly from the macOS command line and loading http://www.syfy.com/theexpanse/enterthefuture/inside. Do we still need these log messages?

[DEBUG SHUTDOWN] ShutdownDecoder: decoder: 'apple coremedia decoder' (0x16a864820) flush:1
[DEBUG SHUTDOWN] Shutdown: reader=0x16b259000 shutdown demuxer=0x16b240710
[DEBUG SHUTDOWN] Shutdown: pool=0x1423fad90 count=2
[DEBUG SHUTDOWN] ShutdownDecoder: decoder: 'apple coremedia decoder' (0x16b1561e0) flush:1
[DEBUG SHUTDOWN] Shutdown: reader=0x16b350000 shutdown demuxer=0x164014e20
[DEBUG SHUTDOWN] Shutdown: pool=0x142305cd0 count=2
[DEBUG SHUTDOWN] operator(): pool=0x1423fad90 shutdown=true count=1
[DEBUG SHUTDOWN] operator(): pool=0x142305cd0 shutdown=true count=1
[DEBUG SHUTDOWN] operator(): pool=0x1423fad90 shutdown=true count=0
[DEBUG SHUTDOWN] TearDownDecoders: reader=0x16b259000 shut down audio task queue
[DEBUG SHUTDOWN] TearDownDecoders: reader=0x16b259000 shut down video task queue
...

	John Lin [:jhlin][:jolin] Assignee
	Comment 25 • 7 years ago

(In reply to Chris Peterson [:cpeterson] from comment #24)
> Jon, I see lots of these "DEBUG SHUTDOWN" log messages when running Nightly
> from the macOS command line and loading
> http://www.syfy.com/theexpanse/enterthefuture/inside. Do we still need these
> log messages?
> 
> [DEBUG SHUTDOWN] ShutdownDecoder: decoder: 'apple coremedia decoder'
> (0x16a864820) flush:1
> [DEBUG SHUTDOWN] Shutdown: reader=0x16b259000 shutdown demuxer=0x16b240710
> [DEBUG SHUTDOWN] Shutdown: pool=0x1423fad90 count=2
> [DEBUG SHUTDOWN] ShutdownDecoder: decoder: 'apple coremedia decoder'
> (0x16b1561e0) flush:1
> [DEBUG SHUTDOWN] Shutdown: reader=0x16b350000 shutdown demuxer=0x164014e20
> [DEBUG SHUTDOWN] Shutdown: pool=0x142305cd0 count=2
> [DEBUG SHUTDOWN] operator(): pool=0x1423fad90 shutdown=true count=1
> [DEBUG SHUTDOWN] operator(): pool=0x142305cd0 shutdown=true count=1
> [DEBUG SHUTDOWN] operator(): pool=0x1423fad90 shutdown=true count=0
> [DEBUG SHUTDOWN] TearDownDecoders: reader=0x16b259000 shut down audio task
> queue
> [DEBUG SHUTDOWN] TearDownDecoders: reader=0x16b259000 shut down video task
> queue
> ...

 Yes, the debug messages were added to diagnose intermittent shutdown time-outs on Android and are still needed. If you find it annoying, perhaps we could make it Fennec only?

	Chris Peterson [:cpeterson]
	Comment 26 • 7 years ago

(In reply to John Lin [:jolin][:jhlin] from comment #25)
>  Yes, the debug messages were added to diagnose intermittent shutdown
> time-outs on Android and are still needed. If you find it annoying, perhaps
> we could make it Fennec only?

They're not so bad. :) I don't think we need to change them. Few people Firefox Nightly from the command line and that website just happened to instantiate a lot of video decoders. For comparison, YouTube only instantiates one per page.

	Pulsebot
	Comment 29 • 7 years ago

Pushed by cpearce@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/f5e1970ca337
Backed out changeset 6524c2d721db. r=backout

	Chris Pearce [:cpearce (Not reading bugmail)]
	Comment 30 • 7 years ago

I backed out the logging, as it's annoying on non-Android, and no-one is looking at the failure. We can re-land the logging (Android only please) when someone has cycles to look at this again.

	Stefan Hindli [:stefan_hindli]
	Comment 31 • 7 years ago
bugherder

https://hg.mozilla.org/mozilla-central/rev/f5e1970ca337

	Firefox Bug Husbandry Bot
	Comment 32 • 6 years ago

Re-triaging per https://bugzilla.mozilla.org/show_bug.cgi?id=1473195

Needinfo :susheel if you think this bug should be re-triaged.

	BMO Automation
	Comment 41 • 4 years ago

We have completed our launch of our new Firefox on Android. The development of the new versions use GitHub for issue tracking. If the bug report still reproduces in a current version of [Firefox on Android nightly](https://play.google.com/store/apps/details?id=org.mozilla.fenix) an issue can be reported at the [Fenix GitHub project](https://github.com/mozilla-mobile/fenix/). If you want to discuss your report please use [Mozilla's chat](https://wiki.mozilla.org/Matrix#Connect_to_Matrix) server https://chat.mozilla.org and join the [#fenix](https://chat.mozilla.org/#/room/#fenix:mozilla.org) channel.