How to reproduce:
Play a Youtube video

What happens:
Firefox attempts to use VA-API, and decodes one frame with it, but then falls back to software decoding.

What should happen:
Firefox shouldn't have fallen back to software decoding.

I'm not able to reproduce this myself, but I've had two reports of this from users. Full logs with debug output are available here.

The relevant part of the log is this:

[RDD 18855: MediaPDecoder #2]: D/Dmabuf DMABufSurfaceYUV::CreateYUVSurface() UID 1 from desc
[RDD 18855: MediaPDecoder #2]: D/Dmabuf DMABufSurfaceYUV::UpdateYUVData() UID 1
[RDD 18855: MediaPDecoder #2]: D/Dmabuf     plane 0 size 1920 x 1080 format 20203852
[RDD 18855: MediaPDecoder #2]: D/Dmabuf     plane 1 size 960 x 540 format 38384752
[RDD 18855: MediaPDecoder #2]: D/PlatformDecoderModule Created new VA-API DMABufSurface UID = 1
[RDD 18855: MediaPDecoder #2]: D/PlatformDecoderModule VideoFrameSurfaceVAAPI: creating surface UID = 1
[RDD 18855: MediaPDecoder #2]: D/Dmabuf DMABufSurfaceYUV::VerifyTextureCreation() UID 1
[RDD 18855: MediaPDecoder #2]: D/Dmabuf DMABufSurfaceYUV::CreateEGLImage() UID 1 plane 0
[RDD 18855: MediaPDecoder #2]: D/Dmabuf   Success.
[RDD 18855: MediaPDecoder #2]: D/Dmabuf DMABufSurfaceYUV::CreateEGLImage() UID 1 plane 1
[RDD 18855: MediaPDecoder #2]: D/Dmabuf   Success.
[RDD 18855: MediaPDecoder #2]: D/Dmabuf DMABufSurfaceYUV::ReleaseEGLImages() UID 1
[RDD 18855: MediaPDecoder #2]: D/PlatformDecoderModule VideoFrameSurfaceVAAPI: VAAPI locking dmabuf surface UID = 1
[Child 18722: MediaSupervisor #1]: D/PlatformDecoderModule Sandbox RDD decoder supports requested type
[RDD 18923: Main Thread]: D/PlatformDecoderModule PDMInitializer, Init PDMs in RDD process
[RDD 18923: Main Thread]: D/Dmabuf Using DRM device /dev/dri/renderD128
[RDD 18923: Main Thread]: D/Dmabuf nsDMABufDevice::IsDMABufVAAPIEnabled: EGL 1 media_ffmpeg_vaapi_enabled 1 CanUseHardwareVideoDecoding 1 XRE_IsRDDProcess 1
[RDD 18923: Main Thread]: D/Dmabuf nsDMABufDevice::IsDMABufVAAPIEnabled: EGL 1 media_ffmpeg_vaapi_enabled 1 CanUseHardwareVideoDecoding 1 XRE_IsRDDProcess 1
[RDD 18923: Main Thread]: D/PlatformDecoderModule FFMPEG: Couldn't load function av_frame_get_colorspace
[RDD 18923: Main Thread]: D/PlatformDecoderModule FFMPEG: Couldn't load function av_frame_get_color_range
[RDD 18923: Main Thread]: D/PlatformDecoderModule Agnostic decoder rejects requested type
[RDD 18923: Main Thread]: D/PlatformDecoderModule Agnostic decoder supports requested type
[RDD 18923: Main Thread]: D/PlatformDecoderModule Agnostic decoder supports requested type
[RDD 18923: Main Thread]: D/PlatformDecoderModule Agnostic decoder supports requested type
[RDD 18923: Main Thread]: D/PlatformDecoderModule Agnostic decoder supports requested type
[RDD 18923: MediaSupervisor #1]: D/PlatformDecoderModule FFMPEG: FFmpegVideoDecoder::FFmpegVideoDecoder MIME video/vp9 Codec ID 167
[RDD 18923: MediaSupervisor #1]: D/PlatformDecoderModule FFMPEG: VAAPI is disabled by parent decoder module.

It successfully decodes the frame, and just before importing the frame it crashes (or seems to) but doesn't log anything.

I've tried tracing through the code, but I can't see any obvious cause of this.

Would setting MOZ_DISABLE_RDD_SANDBOX=1 fix this issue? If so, then it would be bug 1751363.
Thank you.

(In reply to Alastor Wu [:alwu] from comment #1)

Would setting MOZ_DISABLE_RDD_SANDBOX=1 fix this issue?

No, that environment variable is already set as it's required for the nvidia-vaapi-driver to function.