H.264 video playback on Wayland (with MOZ_ENABLE_WAYLAND=1) with VA API enabled (gfx.webrender.all:true media.ffmpeg.vaapi.enabled:true) on an nVidia NVS 5400M (GF108M) using the nouveau driver on Debian testing results in corrupted/garbled video, as seen in the attachment. The problem does not appear under xwayland (without MOZ_ENABLE_WAYLAND).

VA API reports the only supported profile is VAProfileNone (presumably because I am missing firmware):
$ vainfo
libva info: VA-API version 1.9.0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/nouveau_drv_video.so
libva info: Found init function __vaDriverInit_1_9
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.9 (libva 2.9.0)
vainfo: Driver version: Mesa Gallium driver 20.2.2 for NVC1
vainfo: Supported profile and entrypoints
VAProfileNone : VAEntrypointVideoProc

This is confirmed by mpv --hwdec=vaapi --hwdec-codecs=all BigBuckBunny_512kb.mp4, which prints:
[ffmpeg/video] h264: No support for codec h264 profile 578

mozregression --pref gfx.webrender.all:true media.ffmpeg.vaapi.enabled:true -a https://archive.org/download/BigBuckBunny_328/BigBuckBunny_512kb.mp4
Last good revision: 9498aff64f062c246ccdec5013236bd03470b609
First bad revision: 003b4ce4b7e1adca10b2d8600e89f7263af1887e
Pushlog: https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=9498aff64f062c246ccdec5013236bd03470b609&tochange=003b4ce4b7e1adca10b2d8600e89f7263af1887e

The issue seems superficially similar to Bug 1658772, although that bug did not appear to be Wayland-specific and appeared on a format which the driver claimed to support.