Closed Bug 1790729 Opened 2 years ago Closed 1 year ago

Green corruption in vp8 webrtc video with VA-API (i965_drv_video.so and iHD_drv_video.so)

Categories

(Core :: WebRTC: Audio/Video, defect, P3)

x86_64
Linux
defect

Tracking

()

VERIFIED FIXED
115 Branch
Tracking Status
firefox106 --- disabled
firefox115 --- verified
firefox116 --- verified

People

(Reporter: karlt, Assigned: stransky)

References

(Blocks 1 open bug)

Details

(Keywords: correctness)

Attachments

(5 files)

STR:

  1. Load https://jsfiddle.net/jib1/02j8q6h9/show.
  2. Deselect h264.
  3. Select gUM.

After a few minutes, a blocky greening of the darker areas usually (but not always) slowly develops in the darker areas. After a few tens of seconds the picture resets to expected and immediately begins the gradual greening again.

This has reproduced in Nightly 2022-09-01 and ASAN 2022-09-13.

Decoder: 120 video/VP812,065 frames 90000 max-fs=12288;max-fr=60Discarded packets: 0
Local: 13:10:14 GMT+1200 (New Zealand Standard Time) inbound-rtpReceived 94,397 packets (100304.14 Kb , 252.4 KBps)Lost 0 packetsJitter 0.015
Remote: 13:10:14 GMT+1200 (New Zealand Standard Time) remote-outbound-rtpSent 94,428 packets (100287.03 Kb)

Sometimes the top half of the video becomes mostly green, but the darkest areas are still visible amongst the green. Movement can transfer the green elsewhere.

I've been seeing this since around July on whereby.com, and recently on jit.si and https://jsfiddle.net/jib1/02j8q6h9/show.

Decoder: 120 video/VP811,935 frames 90000 max-fs=12288;max-fr=60Discarded packets: 29
Local: 10:28:42 GMT+1200 (New Zealand Standard Time) inbound-rtpReceived 92,656 packets (98823.10 Kb , 254.7 KBps)Lost 0 packetsJitter 0.011
Remote: 10:28:42 GMT+1200 (New Zealand Standard Time) remote-outbound-rtpSent 110,090 packets (117301.13 Kb)

When the top half becomes green, there is sometimes an irregular flicker. The flicker is fast enough that it looks like lines over the background video, but a screenshot reveals that none of background is identifiable.

This reproduces with a fresh profile, where "media.ffmpeg.vaapi.enabled" is at the default value of false.

It reproduces even in an older profile with
gfx.blacklist.vaapi.failureid FEATURE_FAILURE_GLXTEST_FAILED
gfx.blacklist.vp8.hw-decode.failureid FEATURE_FAILURE_GLXTEST_FAILED
gfx.blacklist.webrtc.hw.acceleration.decode.failureid FEATURE_FAILURE_GLXTEST_FAILED

It reproduced even in a --with-system-libvpx build with libvpx versions both 1.11.0 and 1.12.0.

Despite "media.ffmpeg.vaapi.enabled" false, terminal output includes the following each time the received video size changes, indicating that VA-API is being used:

libva info: VA-API version 1.14.0
libva info: Trying to open /usr/lib64/va/drivers/iHD_drv_video.so
libva info: va_openDriver() returns -1
libva info: Trying to open /usr/lib64/va/drivers/i965_drv_video.so
libva info: Found init function __vaDriverInit_1_12
libva info: va_openDriver() returns 0

Despite "media.ffmpeg.vaapi.enabled" false

media.hardware-video-decoding.force-enabled and media.ffmpeg.vaapi.enabled are the same and mean force-enabling.
VAAPI is enabled by default in Nightly and early Beta.
It can be force-disabled by setting media.hardware-video-decoding.enabled to false.

Keywords: correctness
OS: Unspecified → Linux
Hardware: Unspecified → x86_64
Summary: Green corruption in vp8 webrtc video with VA-API → Green corruption in vp8 webrtc video with VA-API (i965_drv_video.so)

Thanks. I installed intel/media-driver, so that iHD_drv_video.so is used and still get the same gradual greening.

 0:51.49 INFO: b'libva info: VA-API version 1.14.0'
 0:51.49 INFO: b'libva info: Trying to open /usr/lib64/va/drivers/iHD_drv_video.so'
 0:51.49 INFO: b'libva info: Found init function __vaDriverInit_1_14'
 0:51.49 INFO: b'libva info: va_openDriver() returns 0'
Summary: Green corruption in vp8 webrtc video with VA-API (i965_drv_video.so) → Green corruption in vp8 webrtc video with VA-API (i965_drv_video.so and iHD_drv_video.so)
Severity: -- → S3

Karl, is the testcase still valid? I'm getting:

NotFoundError: The object can not be found here.
Receiver 	Sender
0x0 	0x0
0 bps 	0 bps

Thanks.

Flags: needinfo?(karlt)

Ah I see, I need enabled webcam for it.

Flags: needinfo?(karlt)

Karl, can you attach your about:support page?
Also I'm unable to run the fiddle, I'm getting:

'TypeError: can't access property "getUserMedia", navigator.mediaDevice is undefined

Flags: needinfo?(karlt)
Priority: -- → P3

The fiddle is still behaving the same for me. It uses navigator.mediaDevices.getUserMedia().

Flags: needinfo?(karlt)
Attached file about:support (deleted) —

Let's disable VP8/VA-API decoding for WebRTC. I have reports about broken AMD decoding too (while majority of AMD devices may not support VP8 anyway).

It seems to depend on actual web cam device. I tested that on a laptop with Intel 530 gfx and I'm getting VP8 corruption from build-in camera while attached additional USB camera doesn't show it.

Assignee: nobody → stransky
Status: NEW → ASSIGNED
Pushed by stransky@redhat.com: https://hg.mozilla.org/integration/autoland/rev/4064731b3c9d [Linux] Disable VP8 hardware decoding for WebRTC r=alwu
Status: ASSIGNED → RESOLVED
Closed: 1 year ago
Resolution: --- → FIXED
Target Milestone: --- → 115 Branch

(In reply to Martin Stránský [:stransky] (ni? me) from comment #12)

It seems to depend on actual web cam device. I tested that on a laptop with Intel 530 gfx and I'm getting VP8 corruption from build-in camera while attached additional USB camera doesn't show it.

Thanks for investigating, Martin. Reproduction is unreliable for me, so it may be hard to confirm association with a particular cam device.
FWIW the one I was using was "1bcf:28b8 Sunplus Innovation Technology Inc. Integrated_Webcam_HD" according to lsusb.
I have also occasionally seen this issue on remote streams, where I don't know what camera was in use.

I updated the MESA ticket. In short when stream is taken by

Bus 001 Device 004: ID 5986:0708 Bison Electronics Inc. Integrated Camera

on Lenovo T460p and then encoded to VP8 the corruption is visible on all Intel/VA-API devices connected to a call or via. loopback on the device.

What I can think of that is different between cameras are their capabilities. That is, resolution, pixel format and framerate. We convert all frames to I420 and framerate should make no difference to the codec. Is there a particular resolution which this is affected for? You can get the resolution from the videoWidth and videoHeight attrobutes of the video element. Hw encoders often have alignment requirements but this would the first time I hear about it for decoders.

Another thing that is specific to webrtc, since you mention gradual changes, is the lack of keyframes. Typically a key frame is only encoded on request as the receiver knows, and can signal to the sender, when it lost video, and it will never need to seek. Therefore I am thinking you should be able to create a reproducible test case by encoding the camera stream with ffmpeg, using libvpx in realtime mode with no periodic keyframes. Then decoding with the hw decoder should exhibit the same issues? And if it doesn't, well that is valuable information too.

I tested two ways how to capture/record VP8 stream from camera. One with ffmpeg (https://trac.ffmpeg.org/wiki/Capture/Webcam) and one with web test (https://webrtc.github.io/samples/src/content/getusermedia/record/) but none of them shows the corruption.
I wonder how the corrupted webrtc stream is encoded - does it missing keyframes completely for instance?

(In reply to Martin Stránský [:stransky] (ni? me) from comment #19)

I wonder how the corrupted webrtc stream is encoded - does it missing keyframes completely for instance?

It is typical to only have the initial keyframe and then just inter frames for the rest, yes.

Flags: qe-verify+

Reproducible on a 2023-05-15 Nightly build on Ubuntu 22.
Verified as fixed on Firefox 115.0b4(build ID: 20230611180300) and Nightly 116.0a1(build ID: 20230611214645) on Ubuntu 22.

Status: RESOLVED → VERIFIED
Flags: qe-verify+
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: