Green corruption in vp8 webrtc video with VA-API (i965_drv_video.so and iHD_drv_video.so)
Categories
(Core :: WebRTC: Audio/Video, defect, P3)
Tracking
()
People
(Reporter: karlt, Assigned: stransky)
References
(Blocks 1 open bug)
Details
(Keywords: correctness)
Attachments
(5 files)
STR:
- Load https://jsfiddle.net/jib1/02j8q6h9/show.
- Deselect h264.
- Select gUM.
After a few minutes, a blocky greening of the darker areas usually (but not always) slowly develops in the darker areas. After a few tens of seconds the picture resets to expected and immediately begins the gradual greening again.
This has reproduced in Nightly 2022-09-01 and ASAN 2022-09-13.
Decoder: 120 video/VP812,065 frames 90000 max-fs=12288;max-fr=60Discarded packets: 0
Local: 13:10:14 GMT+1200 (New Zealand Standard Time) inbound-rtpReceived 94,397 packets (100304.14 Kb , 252.4 KBps)Lost 0 packetsJitter 0.015
Remote: 13:10:14 GMT+1200 (New Zealand Standard Time) remote-outbound-rtpSent 94,428 packets (100287.03 Kb)
Reporter | ||
Comment 1•2 years ago
|
||
Sometimes the top half of the video becomes mostly green, but the darkest areas are still visible amongst the green. Movement can transfer the green elsewhere.
I've been seeing this since around July on whereby.com, and recently on jit.si and https://jsfiddle.net/jib1/02j8q6h9/show.
Decoder: 120 video/VP811,935 frames 90000 max-fs=12288;max-fr=60Discarded packets: 29
Local: 10:28:42 GMT+1200 (New Zealand Standard Time) inbound-rtpReceived 92,656 packets (98823.10 Kb , 254.7 KBps)Lost 0 packetsJitter 0.011
Remote: 10:28:42 GMT+1200 (New Zealand Standard Time) remote-outbound-rtpSent 110,090 packets (117301.13 Kb)
Reporter | ||
Comment 2•2 years ago
|
||
When the top half becomes green, there is sometimes an irregular flicker. The flicker is fast enough that it looks like lines over the background video, but a screenshot reveals that none of background is identifiable.
Reporter | ||
Comment 3•2 years ago
|
||
This reproduces with a fresh profile, where "media.ffmpeg.vaapi.enabled" is at the default value of false.
It reproduces even in an older profile with
gfx.blacklist.vaapi.failureid FEATURE_FAILURE_GLXTEST_FAILED
gfx.blacklist.vp8.hw-decode.failureid FEATURE_FAILURE_GLXTEST_FAILED
gfx.blacklist.webrtc.hw.acceleration.decode.failureid FEATURE_FAILURE_GLXTEST_FAILED
It reproduced even in a --with-system-libvpx build with libvpx versions both 1.11.0 and 1.12.0.
Despite "media.ffmpeg.vaapi.enabled" false, terminal output includes the following each time the received video size changes, indicating that VA-API is being used:
libva info: VA-API version 1.14.0
libva info: Trying to open /usr/lib64/va/drivers/iHD_drv_video.so
libva info: va_openDriver() returns -1
libva info: Trying to open /usr/lib64/va/drivers/i965_drv_video.so
libva info: Found init function __vaDriverInit_1_12
libva info: va_openDriver() returns 0
Comment 4•2 years ago
|
||
Despite "media.ffmpeg.vaapi.enabled" false
media.hardware-video-decoding.force-enabled and media.ffmpeg.vaapi.enabled are the same and mean force-enabling.
VAAPI is enabled by default in Nightly and early Beta.
It can be force-disabled by setting media.hardware-video-decoding.enabled to false.
Reporter | ||
Comment 5•2 years ago
|
||
Thanks. I installed intel/media-driver, so that iHD_drv_video.so is used and still get the same gradual greening.
0:51.49 INFO: b'libva info: VA-API version 1.14.0'
0:51.49 INFO: b'libva info: Trying to open /usr/lib64/va/drivers/iHD_drv_video.so'
0:51.49 INFO: b'libva info: Found init function __vaDriverInit_1_14'
0:51.49 INFO: b'libva info: va_openDriver() returns 0'
Updated•2 years ago
|
Assignee | ||
Comment 6•2 years ago
|
||
Karl, is the testcase still valid? I'm getting:
NotFoundError: The object can not be found here.
Receiver Sender
0x0 0x0
0 bps 0 bps
Thanks.
Assignee | ||
Comment 8•2 years ago
|
||
Karl, can you attach your about:support page?
Also I'm unable to run the fiddle, I'm getting:
'TypeError: can't access property "getUserMedia", navigator.mediaDevice is undefined
Assignee | ||
Updated•2 years ago
|
Reporter | ||
Comment 9•2 years ago
|
||
The fiddle is still behaving the same for me. It uses navigator.mediaDevices.getUserMedia()
.
Reporter | ||
Comment 10•2 years ago
|
||
Assignee | ||
Updated•2 years ago
|
Assignee | ||
Updated•1 year ago
|
Assignee | ||
Comment 11•1 year ago
|
||
Let's disable VP8/VA-API decoding for WebRTC. I have reports about broken AMD decoding too (while majority of AMD devices may not support VP8 anyway).
Assignee | ||
Updated•1 year ago
|
Assignee | ||
Comment 12•1 year ago
|
||
It seems to depend on actual web cam device. I tested that on a laptop with Intel 530 gfx and I'm getting VP8 corruption from build-in camera while attached additional USB camera doesn't show it.
Assignee | ||
Comment 13•1 year ago
|
||
Updated•1 year ago
|
Comment 14•1 year ago
|
||
Comment 15•1 year ago
|
||
bugherder |
Reporter | ||
Updated•1 year ago
|
Reporter | ||
Comment 16•1 year ago
|
||
(In reply to Martin Stránský [:stransky] (ni? me) from comment #12)
It seems to depend on actual web cam device. I tested that on a laptop with Intel 530 gfx and I'm getting VP8 corruption from build-in camera while attached additional USB camera doesn't show it.
Thanks for investigating, Martin. Reproduction is unreliable for me, so it may be hard to confirm association with a particular cam device.
FWIW the one I was using was "1bcf:28b8 Sunplus Innovation Technology Inc. Integrated_Webcam_HD" according to lsusb.
I have also occasionally seen this issue on remote streams, where I don't know what camera was in use.
Assignee | ||
Comment 17•1 year ago
|
||
I updated the MESA ticket. In short when stream is taken by
Bus 001 Device 004: ID 5986:0708 Bison Electronics Inc. Integrated Camera
on Lenovo T460p and then encoded to VP8 the corruption is visible on all Intel/VA-API devices connected to a call or via. loopback on the device.
Comment 18•1 year ago
|
||
What I can think of that is different between cameras are their capabilities. That is, resolution, pixel format and framerate. We convert all frames to I420 and framerate should make no difference to the codec. Is there a particular resolution which this is affected for? You can get the resolution from the videoWidth and videoHeight attrobutes of the video element. Hw encoders often have alignment requirements but this would the first time I hear about it for decoders.
Another thing that is specific to webrtc, since you mention gradual changes, is the lack of keyframes. Typically a key frame is only encoded on request as the receiver knows, and can signal to the sender, when it lost video, and it will never need to seek. Therefore I am thinking you should be able to create a reproducible test case by encoding the camera stream with ffmpeg, using libvpx in realtime mode with no periodic keyframes. Then decoding with the hw decoder should exhibit the same issues? And if it doesn't, well that is valuable information too.
Assignee | ||
Comment 19•1 year ago
|
||
I tested two ways how to capture/record VP8 stream from camera. One with ffmpeg (https://trac.ffmpeg.org/wiki/Capture/Webcam) and one with web test (https://webrtc.github.io/samples/src/content/getusermedia/record/) but none of them shows the corruption.
I wonder how the corrupted webrtc stream is encoded - does it missing keyframes completely for instance?
Comment 20•1 year ago
|
||
(In reply to Martin Stránský [:stransky] (ni? me) from comment #19)
I wonder how the corrupted webrtc stream is encoded - does it missing keyframes completely for instance?
It is typical to only have the initial keyframe and then just inter frames for the rest, yes.
Updated•1 year ago
|
Comment 21•1 year ago
|
||
Reproducible on a 2023-05-15 Nightly build on Ubuntu 22.
Verified as fixed on Firefox 115.0b4(build ID: 20230611180300) and Nightly 116.0a1(build ID: 20230611214645) on Ubuntu 22.
Description
•