User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0

Steps to reproduce:

I have a laptop with two GPUs:

1. iGPU identified as: AMD RAVEN (DRM 3.38.0, 5.8.0-rc4-10-mainline, LLVM 10.0.0) - supports VP9
2. dGPU identified as Radeon RX 560 Series (POLARIS11, DRM 3.38.0, 5.8.0-rc4-10-mainline, LLVM 10.0.0) - doesn't support VP9

I tried playing a youtube video in VP9 and noticed in the log that VAAPI decode is not used.

Actual results:

Log: https://pastebin.com/rdD3UH8B
As seen in the log, for some reason, FF tries to use my dGPU for video decoding and fails because VP9 is not supported.

Expected results:

I would expect that the iGPU is used by default or maybe it tries to use all GPUs before going back to CPU decoding. During this test DRI_PRIME=1 or similar is NOT set.

The report says FF 78 but I tested this issue on FF 80 Nightly and Gnome Wayland.

Bugbug thinks this bug should belong to this component, but please revert this change in case of error.

Which graphics card is used to render your desktop? If your monitor is attached to the RX 560, this is the expected behavior.

about:support shows AMD RAVEN as the one GPU being used and doesn't show any other GPU (Windows shows both, iGPU as active). I am guessing the laptop screen is connected to the iGPU since that is usually the case in laptops with switchable graphics.

Which one GPU is mentioned on about:support page?

The integrated one - AMD RAVEN.

I have noticed, that for some reason the dGPU is not always used, sometimes the iGPU is used and works correctly. I will test this with mpv, if it is a Firefox issue or VAAPI? issue.

On both my laptops iGPU's (as VAAPI devices) are available at /dev/dri/renderD128 so I am guessing it is usually the same everywhere. If FF used the GPU at /dev/dri/renderD128 by default it would solve the issue. Or in another words, FF should try the device at /dev/dri/renderD128 first.

I noticed today that this issue is only present on FF Nightly. FF Stable works correctly / uses the iGPU.

Please open about:support in Nightly, click on "Copy text to clipboard" and paste it here.
Which VAAPI prefs do you have enabled? Only media.ffmpeg.vaapi.enabled, or media.ffmpeg.vaapi-drm-display.enabled as well?
Which env vars do you use to start Nightly, if any?

Both are enabled, output from env looks like this: https://pastebin.com/c0nghMU4

Oh, and it looks like enabling only "media.ffmpeg.vaapi.enabled" and leaving "media.ffmpeg.vaapi-drm-display.enabled" disabled fixes this issue. But only on Wayland. On Xorg the issue remains the same.

X11 might need bug 1588904 comment 8.

(In reply to mthw0 from comment #8)

On both my laptops iGPU's (as VAAPI devices) are available at /dev/dri/renderD128 so I am guessing it is usually the same everywhere.

Ordering of /dev/dri/renderDNNN nodes is not stable. On one boot iGPU may be renderD128, on another boot it may become renderD129 while renderD128 is assigned to dGPU.