User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.61 Safari/537.36

Steps to reproduce:

1. play hls video with closed caption or subtitle.

Actual results:

Subtitle/Closed Caption not visible. It was working fine on Firefox 91 but stopped working on 92.

Expected results:

It should display subtitle/closed caption

The Bugbug bot thinks this bug should belong to the 'Core::Audio/Video: Playback' component, and is moving the bug to that component. Please revert this change in case you think the bot is wrong.

Can you please link or upload a specific example that reproduces the issue?

Hi Jon

Here is the link event I have created for you. https://b.rev-qa.vbrick.com/#/events/57678cbf-0f0b-456e-a763-0ed65d8ac756

You will see a guest page at first. Please provide a name and email id and you can watch the event. When you hover on the player, you will see cc button with languages.. when you select any language, you should see live subtitles.

if you run the event on FireFox 91.. you will see the subtitles but not on 92.

I keep this event running till 02-oct 10 AM (EST)

Please let me know if you need any further info.

I keep this event running till 02-oct 10 AM (EST)

It's unlikely I'll be able to look at this by then. Assuming this doesn't occur with static files, the fastest way to get this addressed would be for you to run mozregression to determine the when exactly this behavior changed. Would you be able to do that?

I have the same issue. I did run mozregression and could isolate it to the following change:
https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=12b5673aeec96f57766e1472cd10acda8fd0989c&tochange=a11c2bd2de51f69740198b60e470f93c2ad3a617

Thought you might want a look at this one, Alastor

Hi, I wonder if you have any example site or a test case which would let us to debug this issue?
Thank you.

Also, CCed TY per comment5's result.

https://file.interru.io/example/bug_mozilla_1733232/

That's currently the most minimal example I have which consistently fails.
Some (basic) examples fail after a couple of reloads.
I assume that's because of caching and layout complexity.

I can reproduce this, will take a look.

I don't know if this is relevant, but i am on Firefox 95.0 (64bit) on Ubuntu 18.04, and I experience something similar. I am trying to create a video player, and subtitles worked for 2-3 hours while developing it, and suddenly they are gone. It works on chrome 96.0.4664.45 (Official Build) (64-bit).

(In reply to Mathias Köhler from comment #8)

I assume that's because of caching and layout complexity.

I'm finding the same behavior for my layout. Once a particular layout makes captions fail, they stay hidden, even when I remove all styles from the page. I'm not used to a bug behaving that way. Any info as to why that happens would be interesting to me.

I can confirm bug 1702401 regresses this bug. I load the testcase in comment 8, and I don't see any caption frames are generated under the video frame in the frame tree dump. I haven't debug yet, so it's unclear to my why rejecting a dirty flex item's cache can suppress the caption generation.

Alastor, do you know where is the code generating the dom elements (or layout frames) for the captions?

Set release status flags based on info from the regressing bug 1702401

For displaying cues, we first would get the overlay from the video frame, and create a root cue div under that overlay. Then, for each cue, we would calculate their position and append new div under the root cue div.

Here's a bugzilla-hosted testcase, taken from comment 8 (with very-minor simplification).

Some observation:

Enable media.webvtt.debug.logging=true pref, and load testcase 1 (in comment 17). After playing the video, I see

JavaScript error: resource://gre/modules/vtt.jsm, line 1222: InvalidStateError: An attempt was made to use an object that is not, or is no longer, usable

If I play the video again, I see no [vtt] log at all.

More interestingly, after flipping the debug pref layout.flexbox.item-final-reflow-optimization.enabled=false, which resulting more reflow to flex items. I see the following:

JavaScript error: chrome://global/content/elements/videocontrols.js, line 149: InternalError: too much recursion
JavaScript error: chrome://global/content/elements/videocontrols.js, line 149: InternalError: too much recursion
[vtt] Adjust position when 'snap-to-lines' is true.
[vtt] box's top moved from 0 to 268
[vtt] cue 0, top=268, bottom=285, left=0, right=1024, width=1024, height=17
[vtt] === processCues ===
JavaScript error: resource://gre/modules/vtt.jsm, line 1222: InvalidStateError: An attempt was made to use an object that is not, or is no longer, usable
JavaScript error: chrome://global/content/elements/videocontrols.js, line 3296: InternalError: too much recursion
JavaScript error: chrome://global/content/elements/videocontrols.js, line 3296: InternalError: too much recursion

It looks like changing the number of reflow can affect the behavior of videocontrols.js, and can cause js stack to overflow? There was an attempt in bug 1493525 to prevent re-entrance of the module. https://searchfox.org/mozilla-central/rev/41614c2fb53602e9a4c9793afa76f2422c0bf9a2/toolkit/content/widgets/videocontrols.js#921-930. However, I'm not familiar with the code, so I'll stop my investigation here.

Loading https://file.interru.io/example/bug_mozilla_1733232/ in my local debug build, I see a lot of following.

JavaScript error: chrome://global/content/elements/videocontrols.js, line 2277: Error: Please don't trigger reflow. See bug 1493525.

Based on the investigation on comment18, Mike, would you mind to help me take a look of this issue when you have free time?
Thank you.

I think I'm going to redirect this to some folks who are spending more time in videocontrols.js these days. mhowell, do you, kpatenio or niklas have a few cycles to help figure out what's going on here?

Oh this is quite a bit deeper than any of the work we've been doing, but I can try.

Per comment 18, the number of reflow can affect the behavior of videocontrols.js, and can cause js stack to overflow?
I am going to move this to video control component for a further investigation.

Bug 1758578 would probably be related with this as well.

I did more investigation today, I feel this bug should probably a layout bug again.

First, in the console, I can see following error

# InvalidStateError: An attempt was made to use an object that is not, or is no longer, usable
processCuesInternal resource://gre/modules/vtt.jsm:1222
processCues resource://gre/modules/vtt.jsm:1319
processCues resource://gre/modules/WebVTTParserWrapper.jsm:51
get chrome://global/content/elements/videocontrols.js:2282
updateReflowedDimensions chrome://global/content/elements/videocontrols.js:2317
handleControlEvent chrome://global/content/elements/videocontrols.js:929
handleEvent chrome://global/content/elements/videocontrols.js:698
// "get <- updateReflowedDimensions <- handleControlEvent <- handleEvent" repeats N times....

So I did a quick workaround in here (videocontrols.js), adding some code to prevent this part from unlimited re-entry.

case "resizevideocontrols":
  if (this.inReflow) {
    return;
  }
  this.inReflow = true;
  // Since this event come from the layout, this is the only place
  // we are sure of that probing into layout won't trigger or force
  // reflow.
  this.reflowTriggeringCallValidator.isReflowTriggeringPropsAllowed = true;
  this.updateReflowedDimensions();
  this.reflowTriggeringCallValidator.isReflowTriggeringPropsAllowed = false;
  this.adjustControlSize();
  this.updatePictureInPictureToggleDisplay();
  this.inReflow = false;
  break;

By doing so, I can see the subtitle showing again, but it would show in the wrong position. But that is not the point, I then observed an interesting thing. For this video, if you apply the above change, and seek to a place where a subtitle is showing (and keep video paused), then I found that there would be endless resizecaption events...

resizecaption event is dispatched here, and I then printed the new child size and the old child size for a comparison.

DD | width=38400, height=15960, old-width=38977, old-height=16200
DD | width=38400, height=15960, old-width=38977, old-height=16200
DD | width=38977, height=16200, old-width=38400, old-height=15960
DD | width=38977, height=16200, old-width=38400, old-height=15960

You can find that those caption div size changes really quickly, which means we call nsVideoFrame::Reflow in a really really quick frequency. That causes a obvious performance degrade, I could feel the UI become very slow and near frozen. I feel that situation shouldn't happen and is probably a root cause of this issue.

We should not fire those events for measuring reflows.

This ensures not to fire dummy events for measuring reflows for example.

Needs a test, maybe Alwu can help with that?

Alastor, can you help me to write a test for this that test that captions actually show up? I'm not familiar with the relevant tests.

Sure, keep my NI, I will work on a test later. Thank for help!

NI myself as a reminder for uplifting patches to Fx100.

https://hg.mozilla.org/mozilla-central/rev/a6ee0bac48bd
https://hg.mozilla.org/mozilla-central/rev/e54b0ef75884

Awesome. Can confirm that this is fixed for us in the firefox nightly!

Seems like this is targeting Firefox 101 release. Any chance that this could be made available with Firefox 100?

Thanks!

Comment on attachment 9270792 [details]
Bug 1733232 - Use reflow callbacks to properly track video control / caption sizes. r=dholbert

Beta/Release Uplift Approval Request

• User impact if declined: Subtitle would fail to display and it cause CPU performance degrade (too many events)
• Is this code covered by automated tests?: Yes
• Has the fix been verified in Nightly?: No
• Needs manual test from QE?: Yes
• If yes, steps to reproduce: Test case in comment17
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): This change didn't introduce a new feature, it's about better handling the resize event to avoid notifying non-necessary events.
• String changes made/needed:

Comment on attachment 9270792 [details]
Bug 1733232 - Use reflow callbacks to properly track video control / caption sizes. r=dholbert

Approved for 100.0b5

IF we uplift this we also need to uplift bug 1764212.

https://hg.mozilla.org/releases/mozilla-beta/rev/68ba2f5407cd
https://hg.mozilla.org/releases/mozilla-beta/rev/f42d2fe8fc3c

I've reproduced the issue on Firefox 99.0.1 and verified the fix on Beta 100.0b5 and Nightly 101.0a1.

Copying crash signatures from duplicate bugs.