We should support Pipewire out of the box in WebRTC. When Wayland session is used (and Firefox does not need to run with Wayland backend), screen sharing is generally broken.

This involves to ship pipewire library wrapper and pipewire headers.

Just so you're aware, we've started working on updating the version of libwebrtc we have in tree (tracked by Bug 1654112) so there will definitely be some churn in the desktop capture code. I'm not sure if that will have a large impact on the work you're planning here.

(In reply to Dan Minor [:dminor] from comment #1)

Just so you're aware, we've started working on updating the version of libwebrtc we have in tree (tracked by Bug 1654112) so there will definitely be some churn in the desktop capture code. I'm not sure if that will have a large impact on the work you're planning here.

Dan, thanks for heads-up. I have the patches ready but it can wait until the update is finished.
Also the pipewire headers and library wrapper should be independent on it unless you also want to pull that parts.

I'll file the patches anyway to have it tracked somewhere. It can be then updated according to the update.

Might be worth mentioning this upstream patch that has been in the works for a while. This patch attempts to reduce the number of dialogs that occur when screensharing with pipewire. After this lands, I would expect that popular vendor patches for supporting pipewire 0.3 will be introduced as well.

https://webrtc-review.googlesource.com/c/src/+/160649/33#message-a83e8959e03a274642ca2ce1abb9ea0099f08097

What's the status on this? For end users like me, do we need to manually patch in the stuff listed in bugs 1654112, 1672945, 1672947, 1672948, 1672987, and 1672989, or wait for proper upstream support with regards to webRTC over Pipewire?

Thanks!

What's the status on this?

It's still being worked on. Until it's stable and enabled if official builds, the only way to get screen sharing on Linux is to manually apply the patches.

I've heard some people using the Fedora firefox binaries, which seem to be already-patched.

Finally, on some scenarios, pushing the video onto a virtual webcam device works (if you can stream a webcam on whatever site you're using).

(In reply to Hugo Osvaldo Barrera from comment #7)

What's the status on this?

It's still being worked on. Until it's stable and enabled if official builds, the only way to get screen sharing on Linux is to manually apply the patches.

I've heard some people using the Fedora firefox binaries, which seem to be already-patched.

Finally, on some scenarios, pushing the video onto a virtual webcam device works (if you can stream a webcam on whatever site you're using).

We are trying to keep track of distro patches here...
https://github.com/emersion/xdg-desktop-portal-wlr/wiki/Screencast-Compatibility#webrtc-aka-firefoxchromium

I think you're referring to v4l2loopback as the "virtual webcam" option, in case anyone is searching for more info. OBS supports sharing video using it with a plugin called obs-v4l2sink, and on wlroots based compositors, wf-recorder has support too.

Nico, any opinions on whether we should take this before or after we do the libwebrtc update? I'm leaning towards taking it now because I know this is causing problems for people and it doesn't look like it would be too difficult to rebase later on, especially if related changes are landing upstream.

dminor: +1, I am for taking it now. It may need some reformatting to meet upstream code style.

Try for the latest patches: https://treeherder.mozilla.org/#/jobs?repo=try&revision=07eb1c937a3448622db0025a2a2ebe1566bdb77c
The 0.3 PW patches are heading to upstream so the affected files at https://phabricator.services.mozilla.com/D94589 can be imported from upstream then.

Let's tracks PW issues here.

I using PipeWire for camera access also in scope?

(In reply to Jan Alexander Steffens [:heftig] from comment #13)

I using PipeWire for camera access also in scope?

I'm not sure what do you mean, can you explain please?

PipeWire can provide access to cameras, which means the sandbox could be tightened as Firefox no longer needs to access V4L2 devices itself.

Some context: a camera portal was introduced in https://github.com/flatpak/xdg-desktop-portal/pull/331 but IIUC the ui is still missing: https://github.com/flatpak/xdg-desktop-portal/issues/414

I'm sure we want to use V4L2 device as we want to process whole recording on GPU (va-api encoding).

Pipewire apparently allow that - Mutter uses pipewire for screen sharing and on supported configurations (currently only intel on wayland) shares buffers via dmabuf. I'm not sure if Firefox already uses dmabuf import for that, but there is a MR for OBS studio to allow direct importing of dmabuf pipewire stream: https://github.com/obsproject/obs-studio/pull/2484

For the record, one of the main benefits of using pipewire is that multiple applications can have simultaneous access to the camera, not only one application, similar to how things used to work for audio before pulseaudio came up. Access management is AFAIK a nice by-product.

Short follow up: this apparently hasn't been implemented anywhere yet (with obs studio flatpak variant being a hot contender to start IMO), but there's a simple example script: https://gitlab.gnome.org/-/snippets/762

So I guess this is well on topic and should be considered - but maybe let some other apps make the first step :)

What are the plans for adding these WebRTC Pipewire changes to the ESR branch?

(In reply to Charles Robertson from comment #20)

What are the plans for adding these WebRTC Pipewire changes to the ESR branch?

None. Will be in next ESR line which is 91.

Currently when using Pipewire for WebRTC, the mouse cursor is not showing in screen sharing. This function should be enabled by default, or a config switch should be added.

Related: https://gitlab.gnome.org/GNOME/mutter/-/issues/1184

The cursor renders when screesharing [with pipewire] for me using sway.

I don't think it's a webrtc or pipewire issue. Maybe it's due to mutter or xdg-desktop-portal-gtk.

The screen cast portal implements a parameter for specifying a cursor mode:
https://github.com/flatpak/xdg-desktop-portal/blob/0a082d7ddc9b6a2c322e38ea2240eac15bb19248/data/org.freedesktop.portal.ScreenCast.xml#L256

Not all portal implementations implement all cursor modes though.
In xdg-desktop-portal-wlr, we support embedded and hidden, but not metadata:
https://github.com/emersion/xdg-desktop-portal-wlr/blob/6cc3a017417fa77fd9f3dd4fe230a9caf10418fb/src/screencast/screencast.c#L281

I would have guessed that the gtk portal supports at least those two modes, but maybe not.

WebRTC asks for embedded first, and then hidden if embedded isn't advertised...
https://webrtc.googlesource.com/src/+/refs/heads/main/modules/desktop_capture/linux/base_capturer_pipewire.cc#1035

I am actually using kwin in wayland. I added a comment here because other software like obs-studio, which also uses kwin and pipewire screen capture, can get the cursor with no issues. So I think there is a switch somewhere and webrtc of firefox should try to turn it on and capture the cursor by default.

Maybe someone can check the implementation of obs-studio.

Kwin may not be advertising the availability of the embedded cursor mode, while obs may ignore the advertisement and try to invoke it anyway. The linked code for WebRTC is what is used in FF and it correctly supports embedded cursors for portal implementations that advertise the functionality. I would recommend raising a bug with the kwin portal implementation you use.

I want to report to this tracker again about mouse cursor in Firefox screencast,

1. Dan Shick suggested that we should check kwin implementation about whether.
   I didn't get an answer from kwin group yet, but by just looking at their code it seems it has been implemented,

https://github.com/KDE/kwin/blob/af4c37c09521e46d1aa470fcf63f4730a6905b78/src/plugins/screencast/screencastmanager.cpp

2. I found that in the latest version of chromium, screencasting using pipewire and webrtc shows the web cursor correctly in kde wayland. So I believe that there is still something that can be done on the firefox side.

I also confirmed that embedded is advised by xdg-desktop-portal-kde

https://github.com/KDE/xdg-desktop-portal-kde/blob/02ffa42fec8d8f86b09ead8585ff81821730cd09/src/screencast.h

It seems that libwebrtc used by firefox,

https://github.com/mozilla/gecko-dev/blob/master/third_party/libwebrtc/modules/desktop_capture/linux/base_capturer_pipewire.cc

is outdated and does not contain the detection of cursor mode. The new version from google, as pointed out by Dan Shick

https://webrtc.googlesource.com/src/+/refs/heads/main/modules/desktop_capture/linux/base_capturer_pipewire.cc#1035

does contain the detection. So I hope the mozilla developer can upgrade their libwebrtc.

Audio doesn't seem to be shared by Firefox when window is shared.

Tested on:

1. Discord
2. Jitsi Meet

What is expected to happen?
When a app is selected to be shared, the audio from that app should be shared to the tab as well (or should have a option to do so)

What happens?
Window is shared but no audio is shared.

Firefox: 95.0.2
Environment: native
xdg-desktop-portall-gtk: 1.12.0
pipewire: 0.3.43

You should open a new bug to discuss further. However this isn't expected to work currently and there is no obvious way to make this work with current Wayland + PipeWire/Pulseaudio protocols. I have raised this issue with the PipeWire developers and they seemed to believe that audit streams and windows shouldn't be linked. So according to them sharing "the audio from [the selected window]" is a meaningless question.

https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/408

Sorry, there was a problem with the detection of inactive users. I'm reverting the change.