James Cheng[:JamesCheng] Reporter
	Description • 9 years ago

We have 
|mInfo.mAudio.mIsRenderedExternally| and |mInfo.mVideo.mIsRenderedExternally| two flags in MDSM to indicate if the audio and video sample should be rendered externally in other process through GMP Rendering API(Bug 1146796).

But these two flag are not sufficient to determine if we should request the sample from reader with decoded data or demuxed-only data.

How could we determine the request mode?

Open this bug for discussing the policy.

	Jean-Yves Avenard [:jya]
	Comment 1 • 9 years ago

Why aren't those flags not sufficient ?

Could imagine that we only set them when the HW will do both decryption+decoding and as such, we reader will always output demuxed-only data.

I would change the name of mIsRenderedExternally to be more descriptive that really it can be managed raw.

	JW Wang [:jwwang] [:jw_wang]
	Comment 2 • 9 years ago

IIRC, TV would like to use its own renderer to control A/V sync to render decoded data. So we have a use case where the renderer wants decoded data instead of demuxed ones.

	James Cheng[:JamesCheng] Reporter
	Comment 3 • 9 years ago

Hi Kilik, how do you think?

	Kilik Kuo [:kikuo] (inactive)
	Comment 4 • 9 years ago

As far as I know in current TV partner's media architecture.
For either EME or non-EME media playback, in order to guarantee smooth playback experience, they always use HW (decrypt)+decode+render.  Demuxing is done in software side.

For now, I'm not sure on TV if there's a case what external consumers want is to render decoded data only.
But for non-TV, this may happen.

	Kilik Kuo [:kikuo] (inactive)
	Comment 5 • 9 years ago

Since mIsRenderedExternally is referred from CDM's capability, it implies decrypt+decode+render already.
We may need to consider both the following cases, 

1. External (decode+render) ==> Hard to find a suitable name, Playback?

2. External Render(render) ==> This might need other APIs to query capability of equipments, e.g. home cinema equipment (wireless display & amplifier that only receive decoded data to render, and there's a synchronization protocol between these equipments themselves.)

I don't know if there's any APIs to query these kind of capability against external devices except EME/CDM cases.  Or maybe I'm just worried too much because case 2 seems rare.

	Jean-Yves Avenard [:jya]
	Comment 6 • 9 years ago

In anycase, I don't understand why this should be stored in the TrackInfo object; it has nothing to do with the media and all to do with the client.

mIsRenderedExternally shouldn't be there.

	Kilik Kuo [:kikuo] (inactive)
	Comment 7 • 9 years ago

(In reply to Jean-Yves Avenard [:jya] from comment #6)
> In anycase, I don't understand why this should be stored in the TrackInfo
> object; it has nothing to do with the media and all to do with the client.
> 
> mIsRenderedExternally shouldn't be there.

The capability information is retrieved from CDMProxy and while I was fixing Bug 1188812, I tended to store these information to identify whether the media track will be handled by external consumer or not. That's a reason why I chose MediaInfo.  Another reason was that I preferred not letting MDSM knows about CDMProxy but this may not be an important concern.

If MDSM get the CDMProxy from MediaDecoder::GetCDMProxy and we definitely can make the decision of creating different type of sinks and configuring MFR against requesting different media data type by certain policy in MDSM.  I think this is ok too.

Does it sound reasonable to you ?

	Jean-Yves Avenard [:jya]
	Comment 8 • 9 years ago

(In reply to Kilik Kuo [:kikuo][:kilikkuo] from comment #7)
> (In reply to Jean-Yves Avenard [:jya] from comment #6)
> > In anycase, I don't understand why this should be stored in the TrackInfo
> > object; it has nothing to do with the media and all to do with the client.
> > 
> > mIsRenderedExternally shouldn't be there.
> 
> The capability information is retrieved from CDMProxy and while I was fixing
> Bug 1188812, I tended to store these information to identify whether the
> media track will be handled by external consumer or not. That's a reason why
> I chose MediaInfo.  Another reason was that I preferred not letting MDSM
> knows about CDMProxy but this may not be an important concern.
> 
> If MDSM get the CDMProxy from MediaDecoder::GetCDMProxy and we definitely
> can make the decision of creating different type of sinks and configuring
> MFR against requesting different media data type by certain policy in MDSM. 
> I think this is ok too.
> 
> Does it sound reasonable to you ?

It does, but you'll want JW opinion on how that can be done so that we don't break the threading model and don't reintroduce a need for the MediaDecoder's monitor. 
Could probably use a mirror for that.

	Ralph Giles (:rillian)
	Comment 9 • 9 years ago

Is this still pending, James? We think the code for this has landed.

	James Cheng[:JamesCheng] Reporter
	Comment 10 • 9 years ago

Hi Ralph,

According to Bug 1194652 Comment 4 described,

this bug is intended to discuss the mechanism(policy) who should invoke the SetDemuxOnly API for any external decoding+rendering cases.

But I think it's not in priority currently.

Thanks

	Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)
	Comment 11 • 8 years ago

Mass change P2 -> P3