+++ This bug was initially created as a clone of Bug #1780962 +++

Steps to reproduce:

1. Run KDE Wayland session under openSUSE Tumbleweed using kernel nouveau drivers
2. Enable VAAPI setting media.ffmpeg.vaapi.enabled to true
3. Watch a H264 video such as https://archive.org/details/BigBuckBunny_328

Actual results:

Video shows corrupt contents with a blue tone eventually freezes Firefox and whole system requiring power button poweroff.
Expected results:

Show video either using VAAPI or fallback to ffmpeg decoding.

This is actually a regression. Issue was fixed for couple of months and somehow resurfaced. Running mozregression GUI shows the following result:

Bug 1812030 - Fix pref observers in nsCookieInjector. r=timhuang

• Fix typo in IsEnabledForCurrentPrefState leading to component not enabling if
  one of the service modes was set to disabled.
• Clean up pref observer code by moving list of prefs to array.

Differential Revision: https://phabricator.services.mozilla.com/D167664

Martin, any idea on this issue? Bug 1812030 doesn't look like the root cause for the regression. Thank you!

bug 1812030 comment 5 merged 2023-01-25.

mozregression --good 2023-01-24 --bad 2023-01-26
https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=ba77054848c41f869d9d6a8e1d2c6e02c39dc34a&tochange=4af274d4ee613437631074174934b5739d002880
contains

52fa6a56b687c22c119bcce966caae5806f50b5a Emilio Cobos Álvarez — Bug 1812289 - Enable CSD titlebar more by default. r=stransky

8e856dd24cbdffbdb6d55c85d039f15eab32baff stransky — Bug 1811210 [Linux] Enable zero copy for VA-API video playback r=alwu

bug 1824458 comment 2 has an overview of the zero copy / surface copy changes.

2023-01-19 (111): bug 1809162 introduced and enforced surface copy [...]

2023-01-25 (111): bug 1811210 made surface copy a fallback by heuristic[...]

# 0 - force disable
# 1 - force enable
# 2 - default
- name: media.ffmpeg.vaapi.force-surface-zero-copy

(In reply to Ilgaz Öcal from comment #0)
Please test media.ffmpeg.vaapi.enabled=true with media.ffmpeg.vaapi.force-surface-zero-copy=0 and 1 and attach their logs.
Do both behave the same or differently?

$ pip3 install mozregression
$ MOZ_LOG="Dmabuf:5,PlatformDecoderModule:5" ~/.local/bin/mozregression --launch 2023-05-09 --pref media.ffmpeg.vaapi.enabled:true media.ffmpeg.vaapi.force-surface-zero-copy:0 -P stdout -a https://archive.org/details/BigBuckBunny_328 > bug1832133_surface_copy.txt

$ MOZ_LOG="Dmabuf:5,PlatformDecoderModule:5" ~/.local/bin/mozregression --launch 2023-05-09 --pref media.ffmpeg.vaapi.enabled:true media.ffmpeg.vaapi.force-surface-zero-copy:1 -P stdout -a https://archive.org/details/BigBuckBunny_328 > bug1832133_zero_copy.txt

Nouveau is not used for va-api by default. I didn't manage to get it working and I haven't try new releases and it's somehow supposed to be broken.
Let's see if zero copy/surface copy option changes anything but I doubt so.

Ilgaz, does mpv player works in va-api more for you? (run mpv --hwdec=vaapi test_clip on terminal).

The bug has a release status flag that shows some version of Firefox is affected, thus it will be considered confirmed.

about:support output from current nightly:
(In reply to Martin Stránský [:stransky] (ni? me) from comment #4)

Nouveau is not used for va-api by default. I didn't manage to get it working and I haven't try new releases and it's somehow supposed to be broken.
Let's see if zero copy/surface copy option changes anything but I doubt so.

Ilgaz, does mpv player works in va-api more for you? (run mpv --hwdec=vaapi test_clip on terminal).

As I am on a rolling release Linux things change very rapidly. I used VAAPI decoding for months and now mpv complains:
[vo/gpu/vaapi] unsupported VA image format nv12
[vo/gpu] Initializing texture for hardware decoding failed.
with a blue window. I tried (S)mplayer and VLC both had problems.

(In reply to Darkspirit from comment #3)

bug 1812030 comment 5 merged 2023-01-25.

mozregression --good 2023-01-24 --bad 2023-01-26
https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=ba77054848c41f869d9d6a8e1d2c6e02c39dc34a&tochange=4af274d4ee613437631074174934b5739d002880
contains

52fa6a56b687c22c119bcce966caae5806f50b5a Emilio Cobos Álvarez — Bug 1812289 - Enable CSD titlebar more by default. r=stransky

8e856dd24cbdffbdb6d55c85d039f15eab32baff stransky — Bug 1811210 [Linux] Enable zero copy for VA-API video playback r=alwu

---

bug 1824458 comment 2 has an overview of the zero copy / surface copy changes.

2023-01-19 (111): bug 1809162 introduced and enforced surface copy [...]

2023-01-25 (111): bug 1811210 made surface copy a fallback by heuristic[...]

---

# 0 - force disable
# 1 - force enable
# 2 - default
- name: media.ffmpeg.vaapi.force-surface-zero-copy

(In reply to Ilgaz Öcal from comment #0)
Please test media.ffmpeg.vaapi.enabled=true with media.ffmpeg.vaapi.force-surface-zero-copy=0 and 1 and attach their logs.
Do both behave the same or differently?

$ pip3 install mozregression
$ MOZ_LOG="Dmabuf:5,PlatformDecoderModule:5" ~/.local/bin/mozregression --launch 2023-05-09 --pref media.ffmpeg.vaapi.enabled:true media.ffmpeg.vaapi.force-surface-zero-copy:0 -P stdout -a https://archive.org/details/BigBuckBunny_328 > bug1832133_surface_copy.txt

$ MOZ_LOG="Dmabuf:5,PlatformDecoderModule:5" ~/.local/bin/mozregression --launch 2023-05-09 --pref media.ffmpeg.vaapi.enabled:true media.ffmpeg.vaapi.force-surface-zero-copy:1 -P stdout -a https:/ /archive.org/details/BigBuckBunny_328 > bug1832133_zero_copy.txt

I have tested both and still same glitches are displayed however please note that I noticed that this system's (openSUSE Tumbleweed rolling release) vaapi support recently became problematic. I am attaching the generated log files.

Problem exists.
output of MOZ_LOG="Dmabuf:5,PlatformDecoderModule:5" ~/.local/bin/mozregression --launch 2023-05-09 --pref media.ffmpeg.vaapi.enabled:true media.ffmpeg.vaapi.force-surface-zero-copy:0 -P stdout -a https://archive.org/details/BigBuckBunny_328 > bug1832133_surface_copy.txt

Problem exists.
output of MOZ_LOG="Dmabuf:5,PlatformDecoderModule:5" ~/.local/bin/mozregression --launch 2023-05-09 --pref media.ffmpeg.vaapi.enabled:true media.ffmpeg.vaapi.force-surface-zero-copy:1 -P stdout -a https://archive.org/details/BigBuckBunny_328 > bug1832133_zero_copy.txt

From the log:

 0:45.36 INFO: b'[RDD 23726: MediaPDecoder #1]: D/Dmabuf DMABufSurfaceYUV::ImportPRIMESurfaceDescriptor() UID 1'
 0:45.36 INFO: b'[RDD 23726: MediaPDecoder #1]: D/Dmabuf     plane 0 size 416 x 240 format 20203852'
 0:45.36 INFO: b'[RDD 23726: MediaPDecoder #1]: D/Dmabuf     plane 1 size 208 x 120 format 20203852'
 0:45.36 INFO: b'[RDD 23726: MediaPDecoder #1]: D/Dmabuf     plane 2 size 104 x 60 format 38385247'
 0:45.36 INFO: b'[RDD 23726: MediaPDecoder #1]: D/Dmabuf     plane 3 size 52 x 30 format 38385247'

That doesn't look like correct NV12 format which is supposed to contain only 2 planes (Y and UV).