This is pre-requirement before implementing time stretching in bug 1517199.

Following paragraphs are copied from :padenot.

HTMLMediaElement can output to a MediaStream. For now it uses DecodedStream, which is push-based. Every time it wants to write, it pushes all of its decoded audio queue to the MTG. The MTG then pulls this data little by little.

If we do the time stretching when pushing, there will be a high latency between playbackRate changes and the playback rate actually changing. This, but for volume changes, has been the source of vocal complaints in the past.

It’s also the last source of the MTG that is push based. Moving it to be pull-based would be nice, this fits in well with our future plans of removing AudioStream and associated machinery.

There is however quite a few bits before we can do the (now easy) task of inserting a time stretcher as a graph input.

Quick update: Currently my patches work well, but got a wpt failure. I'm still debugging that.

Hi, Alastor Wu,
I am looking for a solution for PlaybackRate when html element(audio or video) is connected to createMediaElementSource. Thanks for any suggestion in advance.

(In reply to fahadsharifjutt from comment #2)

Hi, Alastor Wu,
I am looking for a solution for PlaybackRate when html element(audio or video) is connected to createMediaElementSource. Thanks for any suggestion in advance.

That is the purpose of this bug, this bug is a pre-requirement before supporting playback change on the media stream track, which is captured from the media element.

AudioDecoderInputTrack stores the audio data in a SPSC queue which allows MTG to pull data from it by calling AudioDecoderInputTrack::AppendData().

As we will use AudioDecoderInputTrack for audio track later, need to remove some dependency of SourceMediaTrack first.

Current GetElementsAfter() doesn't gurantee that, the result array would still contain the element which end time is earlier or equal to the given time.

Depends on D106041

Comment on attachment 9206825 [details]
Bug 1695265 - part2 : implement a new method on media queue to return an array which endtime are all strictly later than the given time. r?bryce

Revision D107170 was moved to bug 1701992. Setting attachment 9206825 [details] to obsolete.

Because the media event source cleans its listener lazily, that only happens when the source dispatches another new event. If the source didn't dispatch any new event after the listeners revoke themself, then they would be always kept in the media event source as a strong reference.

That doesn't matter in other current usage, but it would cause a memory leak in our case.

• A <-- B : B holds a strong reference to A

DecodedStreamGraphListener <-- lambda as a media event callback <-- ListenerImpl <-- MediaEventSource (mOnEnd/mOnOutput) <-- AudioDecoderInputTrack <-- DecodedStreamGraphListener

So it causes a loop and then all those objects would be leaked. Therefore, we have to break the reference manually when the AudioDecoderInputTrack::DestroyImpl() gets called.

Depends on D106042

Backed out 3 changesets (bug 1695265) for causing crashtest assertion failures.
https://hg.mozilla.org/integration/autoland/rev/64198c31e8e835f54fd858ada6e2fd66a696927a

Push with failures:
https://treeherder.mozilla.org/jobs?repo=autoland&revision=7d2f645a1dc49cf7213f3a823c7a04ce9477b91b&selectedTaskRun=PRiy4x2KQLWU0lVVWZLuFA.0

Failure log:
https://treeherder.mozilla.org/logviewer?job_id=338061055&repo=autoland&lineNumber=14350

https://hg.mozilla.org/mozilla-central/rev/5ac05c27c39b
https://hg.mozilla.org/mozilla-central/rev/3ee41652c232
https://hg.mozilla.org/mozilla-central/rev/43c02933d6bc

== Change summary for alert #30032 (as of Mon, 10 May 2021 04:54:26 GMT) ==

Improvements:

For up to date results, see: https://treeherder.mozilla.org/perfherder/alerts?id=30032