Closed Bug 1787293 Opened 2 years ago Closed 2 years ago

YouTube videos freezing since updating to 104.0 caused by zero copy hw decoding

Categories

(Core :: Graphics, defect, P1)

Firefox 104
defect

Tracking

()

RESOLVED FIXED
106 Branch
Tracking Status
relnote-firefox --- 104+
firefox-esr91 --- unaffected
firefox-esr102 --- unaffected
firefox104 + fixed
firefox105 --- fixed
firefox106 --- fixed

People

(Reporter: bhood, Assigned: sotaro)

References

(Blocks 1 open bug, Regression)

Details

(Keywords: regression)

Attachments

(3 files)

Reddit post:

https://www.reddit.com/r/firefox/comments/wwnxtu/youtube_videos_freezing_since_updating_to_1040/

appears to be the result of recent video performance changes. A workaround exists (setting media.wmf.zero-copy-nv12-texture and gfx.direct3d11.reuse-decoder-device to false), but this needs to be addressed quickly.

I've asked a couple of people for their about:support info

Summary: YouTube videos freezing since updating to 104.0 → YouTube videos freezing since updating to 104.0 caused by MPO

Bug 1780138 seems a good candidate for being the cause

Component: Graphics: WebRender → Graphics

One user (oomcommander) is on NVIDIA GeForce RTX 2070 with 30.0.14.7141

appears to be the result of recent video performance changes. A workaround exists (setting media.wmf.zero-copy-nv12-texture and gfx.direct3d11.reuse-decoder-device to false), but this needs to be addressed quickly.

If gfx.direct3d11.reuse-decoder-device is false, "zero copy hardware decoded video" is also disabled. Then the problem seems to be triggered by "reuse decoder device".

On AMD, reuse decoder device is used long time since Firefox 52 by Bug 1313883. Then if the problem happens since 104, it might be mainly related to NVIDIA GPUs.

On chromium, DXVAVideoDecodeAccelerator reuses d3d11 device(ANGLE Device) when "zero copy hardware decoded video" is enabled.

https://github.com/sotaroikeda/chromium-diagrams/blob/master/media_DXVAVideoDecodeAccelerator.pdf

On some GPUs, "zero copy video" is blocked in gpu_driver_bug_list.json

On gecko, "zero copy video" is blocked on some GPUs based on the gpu_driver_bug_list.json.

It might be better to apply same block list also to FEATURE_REUSE_DECODER_DEVICE.

There was a comment that forcing mp4 addressed the problem. From it, vp9 hardware decoding might not work with "zero copy video" or "reuse decoder device".

https://www.reddit.com/r/firefox/comments/wwnxtu/youtube_videos_freezing_since_updating_to_1040/

On current chromium, D3D11VideoDecoder is used as default hardware video decoder on Windows than DXVAVideoDecodeAccelerator. Then blocklist of "disable_dxgi_zero_copy_video" in gpu_driver_bug_list.json might not be enough.

https://github.com/sotaroikeda/chromium-diagrams/blob/master/media_D3D11VideoDecoder.pdf

D3D11VideoDecoder is blocked on some GPUs. In this case, DXVAVideoDecodeAccelerator is used instead.

(In reply to Sotaro Ikeda [:sotaro] from comment #7)

On current chromium, D3D11VideoDecoder is used as default hardware video decoder than DXVAVideoDecodeAccelerator. Then blocklist of "disable_dxgi_zero_copy_video" in gpu_driver_bug_list.json might not be enough.

One user (oomcommander) is on NVIDIA GeForce RTX 2070 with 30.0.14.7141

The above seems not blocked by "disable_dxgi_zero_copy_video" in gpu_driver_bug_list.json.

oomcomander had no recent crashes in about:crashes which suggests they're not obviously running into bug 1313883.

Blocks: video-perf
Summary: YouTube videos freezing since updating to 104.0 caused by MPO → YouTube videos freezing since updating to 104.0 caused by decoder device reuse
Attached file oomcomander's about:support (deleted) —

I just noticed that oomcomander doesn't have DirectComposition because of mixed refresh rate (144/60).

From Chris Peterson (for reference):

Here's a Reddit post about Facebook and ESPN videos not working in 104 on Windows 10 (but YouTube works for them):
https://www.reddit.com/r/firefox/comments/wycu2p/viedos_on_fb_and_espn_not_playing_yt_works/

I tried reproducing this on GTX 1650 with the 30.0.14.7141 driver and couldn't. But it may be intermittent. I've asked the reporter to update their drivers and see if they can still reproduce.

(In reply to Bob Hood [:bhood] from comment #11)

From Chris Peterson (for reference):

Here's a Reddit post about Facebook and ESPN videos not working in 104 on Windows 10 (but YouTube works for them):
https://www.reddit.com/r/firefox/comments/wycu2p/viedos_on_fb_and_espn_not_playing_yt_works/

The reporter's problem was addressed by https://support.mozilla.org/en-US/kb/fix-video-audio-problems-firefox-windows. Then it seems like a different problem.

https://www.reddit.com/r/firefox/comments/wzoyyz/after_recent_update_videos_are_playing_in_frames/

The above used "AMD Radeon(TM) Vega 8 Mobile Graphics". And the problem was addressed by a way of comment 0.

On AMD, "reuse decoder device" is always enabled. Then the problem should be caused by "zero copy hardware decoded video" on the PC.

From the symptoms(frame drops), the problem might happen by not enough of video buffers. Some video decoders might not allocate enough number of video buffers. "10 + 1 " out put buffers are requested in WMFVideoMFTManager::InitInternal().

With current gecko's architecture, allocated video buffers are released by RemoteDecoderManagerParent::RecvDeallocateSurfaceDescriptorGPUVideo(). It is requested from content process via GPUVideoTextureData::Deallocate(). Release request from content process is not responsive. Then RemoteDecoderManagerParent::mImageMap and RemoteDecoderManagerParent::mTextureMap hold at most around 10 buffers.

https://github.com/sotaroikeda/firefox-diagrams/blob/master/media/dom_media_WMFVideoMFTManager_97.pdf

I wonder if we allocate more buffers at WMFVideoMFTManager::InitInternal(), the problem might be addressed. Though it is not clear yet, if the decoders could actually allocate the buffers as requested.

(In reply to Sotaro Ikeda [:sotaro] from comment #16)

With current gecko's architecture, allocated video buffers are released by RemoteDecoderManagerParent::RecvDeallocateSurfaceDescriptorGPUVideo(). It is requested from content process via GPUVideoTextureData::Deallocate(). Release request from content process is not responsive. Then RemoteDecoderManagerParent::mImageMap and RemoteDecoderManagerParent::mTextureMap hold at most around 10 buffers.

It seems not simple to reduce the above.

Regressed by: 1780138
Attached file Bug 1787293 - Revert Bug 1780138 (deleted) —

Set release status flags based on info from the regressing bug 1780138

I re-think about this bug. I have come to think that it seems better to backout bug 1780138 and address Comment 18. Then re-try re-enabling it on release.

:jrmuizel, how do you think about it?

Flags: needinfo?(jmuizelaar)
Blocks: 1785149

Comment on attachment 9292007 [details]
Bug 1787293 - Revert Bug 1780138

Beta/Release Uplift Approval Request

  • User impact if declined: Some users video has freezes/bad playback
  • Is this code covered by automated tests?: No
  • Has the fix been verified in Nightly?: No
  • Needs manual test from QE?: No
  • If yes, steps to reproduce:
  • List of other uplifts needed: None
  • Risk to taking this patch: Low
  • Why is the change risky/not risky? (and alternatives if risky): This basically reverts the enabling of the zero copy feature.
  • String changes made/needed:
  • Is Android affected?: Yes
Flags: needinfo?(jmuizelaar)
Attachment #9292007 - Flags: approval-mozilla-release?

Comment on attachment 9292007 [details]
Bug 1787293 - Revert Bug 1780138

Beta/Release Uplift Approval Request

  • User impact if declined: Some users video has freezes/bad playback
  • Is this code covered by automated tests?: No
  • Has the fix been verified in Nightly?: No
  • Needs manual test from QE?: No
  • If yes, steps to reproduce:
  • List of other uplifts needed: None
  • Risk to taking this patch: Low
  • Why is the change risky/not risky? (and alternatives if risky): This basically reverts the enabling of the zero copy feature.
  • String changes made/needed:
  • Is Android affected?: Yes
Attachment #9292007 - Flags: approval-mozilla-beta?

Comment on attachment 9292007 [details]
Bug 1787293 - Revert Bug 1780138

Approved for 104.0.1

Attachment #9292007 - Flags: approval-mozilla-release? → approval-mozilla-release+

Comment on attachment 9292007 [details]
Bug 1787293 - Revert Bug 1780138

Approved for 105.0b5

Attachment #9292007 - Flags: approval-mozilla-beta? → approval-mozilla-beta+
Status: NEW → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
Target Milestone: --- → 106 Branch

From Bug 1785149 comment 10, this case seemed to be caused by Windows.

We've got a report that this is a Windows issue:

EDIT: it is indeed a Windows 11 issue for those who are running Windows 11, both preview and beta versions. The updates 22621.586 and 22622.586 fix this.

Hello,
We were unable to reproduce the issue on Firefox 104.
We tested on both Windows 11 and Windows 10, by using Intel 530 GPU, Nvidia GTX 1050 and Nvidia GTX 1650.

Summary: YouTube videos freezing since updating to 104.0 caused by decoder device reuse → YouTube videos freezing since updating to 104.0 caused by zero copy hw decoding
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: