Flatpak Firefox still has slow h264 decoding
Categories
(Release Engineering :: Release Automation: Other, defect)
Tracking
(Not tracked)
People
(Reporter: jrmuizel, Unassigned)
References
(Blocks 1 open bug)
Details
This seems to be caused by us not requiring the ffmpeg-full extension. Other packages like Blender so I'm not sure why we don't.
This was originally going to be fixed by bug 1628407, but that was backed out for reasons not written in the bug and never relanded.
Bug 1639531 added optional support for ffmpeg-full which helps for people who have ffmpeg installed but doesn't for those who don't.
|
||
Comment 1•3 years ago
|
||
My vague recollection was we needed to check with legal. I don't know if that happened or what the outcome was.
|
Reporter | |
Comment 2•3 years ago
|
||
Does depending on ffmpeg-full actually cause us to ship the ffmpeg decoder or is just listed as a dependency?
|
||
Comment 3•3 years ago
|
||
Thoughts:
Who declares to have a h264 patent license by deciding to use the patented technology?
- A user who owns at least one h264 patent license (e.g. by having Windows with included licensed h264 software decoder somewhere installed or by owning an iPhone)
therefore ticks the checkbox to manually install the patented h264 software decoder (ffmpeg).
If the user was licensed, then the software repository has done nothing wrong.
If the user was not licensed, then the software repository might have been like a warez site and the user might have been pirating. - A distribution who offers the user a product that just works.
- A software vendor who offers distributors/distributions a product that just works.
IIUC, the whole supply chain is responsible and must agree by contract to not use the forwarded patented technology unless the user has a license.
If the Firefox flatpak package gets a hard dependency on a patented h264 software decoder, then who made the decision to use the patent?
If it was the distribution, then it needs to buy a license unless it has a contract with the user to not use Firefox unless he owns a h264 patent license.
If it was the sofware vendor (Mozilla), then it needs to buy a license unless it has a contract with the distribution to have a contract with the user to not use Firefox unless he owns a h264 patent license.
If I look at Intel's non-response in https://github.com/intel/media-driver/issues/1276, I am not sure if the Intel hardware decoder I bought as part of my APU actually has a h264 patent license. It might have been the hardware shop who might be obligated to buy a h264 patent license unless it contracts with me to not use my hardware decoder unless I buy a h264 patent license myself.
It might be the case that Windows users are only licensed to use their connected patented h264 hardware decoder device because Windows as a whole is bundled with a h264 patent license due to its bundled h264 software decoder.
- I assume Firefox might be able to contain a patented h264 software decoder (or have a hard dependency on one), if the user must manually tick a checkbox to confirm to own a h264 patent license before being able to play back a h264 video.
Firefox does this already implicitly (not explicitly) by expecting that the user would only install ffmpeg if he owns a h264 patent license. - Could Cisco (which already pays capped license fees) host a "mini-h264-ffmpeg-for-cisco-customers.so" library (reproducible build) for Mozilla like they are hosting OpenH264?
- Could Cisco host a wasm version of ffmpeg for their customers like they are hosting OpenH264?
Ffmpeg would then need to get proper Wasm SIMD instructions for best performance. - Unlikely: Could Cisco or Ubuntu become the publisher of Firefox Flatpak? (Canonical Limited pays h264 patent license fees, but IIUC officially only for hardware that includes Ubuntu and not for the regular user)
- Could Mozilla sell h264 patent license keys to make Firefox use the patented h264 software decoder?
https://codecs.raspberrypi.com/, https://www.microsoft.com/en-us/p/hevc-video-extensions/9nmzlz57r3t7?activetab=pivot:overviewtab
|
|
||
Comment 4•3 years ago
|
||
If the user downloads OpenH264 from Cisco, he becomes licensed to use h264:
If Firefox detects the presence of OpenH264, couldn't it allow the now licensed user to use the ffmpeg h264 software decoder from its hard dependency?
|
|
Reporter | |
Comment 5•3 years ago
|
||
IANAL, but I think the simplest analogy is to a physical device. It's not the user that gets a license to decode h264, it's a decoder that gets distributed that has a license to decode h264.
(In reply to Darkspirit from comment #3)
IIUC, the whole supply chain is responsible and must agree by contract to not use the forwarded patented technology unless the user has a license.
If the Firefox flatpak package gets a hard dependency on a patented h264 software decoder, then who made the decision to use the patent?
The actual binary code of the decoder is thing that needs a patent license. If we're not distributing the decoder we don't need a patent license (like on Windows on macOS). Admittedly, if it's hard dependency it's perhaps it's a bit more nuanced, but you could compare it to a store selling DVDs. Those DVDs have a hard dependency on the user having a decoder to watch them, but the shop selling them doesn't need a patent license. The shop also doesn't care how you decode those DVDs and if someone has paid the patent license fees for the decoder.
If I look at Intel's non-response in https://github.com/intel/media-driver/issues/1276, I am not sure if the Intel hardware decoder I bought as part of my APU actually has a h264 patent license. It might have been the hardware shop who might be obligated to buy a h264 patent license unless it contracts with me to not use my hardware decoder unless I buy a h264 patent license myself.
Intel pays a h264 license fee to be able to sell you a decoder. Whether you can use that decoder without an additional license depends on whether you need to write/distribute any software that does patented things to interface with that decoder. For example, if the hardware decoder did not do entropy decoding and that part needed to be done in a separate piece of software then the distributors of that software would also need a patent license.
It might be the case that Windows users are only licensed to use their connected patented h264 hardware decoder device because Windows as a whole is bundled with a h264 patent license due to its bundled h264 software decoder.
This could be the case.
- I assume Firefox might be able to contain a patented h264 software decoder (or have a hard dependency on one), if the user must manually tick a checkbox to confirm to own a h264 patent license before being able to play back a h264 video.
I don't think this is possible. If we're distributing the decoder we'd need a license. I'm pretty sure we could not distribute a h264 decoder in our macOS builds even though we know that the user always has a h264 decoder provided by the OS available to them.
Firefox does this already implicitly (not explicitly) by expecting that the user would only install ffmpeg if he owns a h264 patent license.
- Could Cisco (which already pays capped license fees) host a "mini-h264-ffmpeg-for-cisco-customers.so" library (reproducible build) for Mozilla like they are hosting OpenH264?
I believe this would be possible but I think it's unlikely to happen.
- Could Cisco host a wasm version of ffmpeg for their customers like they are hosting OpenH264?
Ffmpeg would then need to get proper Wasm SIMD instructions for best performance.
Same as above.
- Unlikely: Could Cisco or Ubuntu become the publisher of Firefox Flatpak? (Canonical Limited pays h264 patent license fees, but IIUC officially only for hardware that includes Ubuntu and not for the regular user)
I think so, but unlikely to happen.
- Could Mozilla sell h264 patent license keys to make Firefox use the patented h264 software decoder?
https://codecs.raspberrypi.com/, https://www.microsoft.com/en-us/p/hevc-video-extensions/9nmzlz57r3t7?activetab=pivot:overviewtab
Probably, but also unlikely to happen.
|
|
||
Comment 6•3 years ago
|
||
(In reply to Jeff Muizelaar [:jrmuizel] from comment #2)
Does depending on
ffmpeg-fullactually cause us to ship the ffmpeg decoder or is just listed as a dependency?
If Flatpak works the same as Snap, then yes:
Snap dependencies seem to be more like build dependencies, not package dependencies.
The "Firefox for Ubuntu" Snap (a squashfs filesystem file) contains its dependencies: libavcodec, libx264.so.155, even an unneeded libx265.so.179. These are files, not symlinks.
Therefore it's good that Canonical and not Mozilla publishes the snap.
$ snap download firefox; unsquashfs firefox*.snap; ls squashfs-root/usr/lib/x86_64-linux-gnu
Fetching snap "firefox"
Fetching assertions for "firefox"
Install the snap with:
snap ack firefox_886.assert
snap install firefox_886.snap
Parallel unsquashfs: Using 4 processors
296 inodes (2820 blocks) to write
[=========================================================================================================================/] 2820/2820 100%
created 251 files
created 43 directories
created 45 symlinks
created 0 devices
created 0 fifos
created 0 sockets
libaom.so.0 libheimntlm.so.0.1.0 libpci.so.3 libssh.so.4.8.4 libvpx.so.6.2.0
libasn1.so.8 libhx509.so.5 libpci.so.3.6.4 libswresample.so.3 libwavpack.so.1
libasn1.so.8.0.0 libhx509.so.5.0.0 libpipewire-0.3.so libswresample.so.3.5.100 libwavpack.so.1.2.1
libavcodec.so.58 libkrb5.so.26 libpipewire-0.3.so.0 libtheoradec.so.1 libwebpmux.so.3
libavcodec.so.58.54.100 libkrb5.so.26.0.0 libpipewire-0.3.so.0.332.0 libtheoradec.so.1.1.4 libwebpmux.so.3.0.1
libavutil.so.56 liblber-2.4.so.2 libroken.so.18 libtheoraenc.so.1 libwebp.so.6
libavutil.so.56.31.100 liblber-2.4.so.2.10.12 libroken.so.18.1.0 libtheoraenc.so.1.1.2 libwebp.so.6.0.2
libcodec2.so.0.9 libldap_r-2.4.so.2 librtmp.so.1 libtwolame.so.0 libwind.so.0
libcurl.so.4 libldap_r-2.4.so.2.10.12 libsasl2.so.2 libtwolame.so.0.0.0 libwind.so.0.0.0
libcurl.so.4.6.0 libmp3lame.so.0 libsasl2.so.2.0.25 libva-drm.so.2 libx264.so.155
libgsm.so.1 libmp3lame.so.0.0.0 libshine.so.3 libva-drm.so.2.700.0 libx265.so.179
libgsm.so.1.0.18 libnghttp2.so.14 libshine.so.3.0.1 libva.so.2 libXt.so.6
libgssapi.so.3 libnghttp2.so.14.19.0 libsnappy.so.1 libva.so.2.700.0 libXt.so.6.0.0
libgssapi.so.3.0.0 libnuma.so.1 libsnappy.so.1.1.8 libva-x11.so.2 libxvidcore.so.4
libhcrypto.so.4 libnuma.so.1.0.0 libsoxr.so.0 libva-x11.so.2.700.0 libxvidcore.so.4.3
libhcrypto.so.4.1.0 libOpenCL.so.1 libsoxr.so.0.1.2 libvdpau.so.1 libzvbi.so.0
libheimbase.so.1 libOpenCL.so.1.0.0 libspeex.so.1 libvdpau.so.1.0.0 libzvbi.so.0.13.2
libheimbase.so.1.0.0 libopus.so.0 libspeex.so.1.5.0 libvpx.so.6 pipewire-0.3
libheimntlm.so.0 libopus.so.0.8.0 libssh.so.4 libvpx.so.6.2 spa-0.2
(In reply to Julien Cristau [:jcristau] from comment #1)
My vague recollection was we needed to check with legal. I don't know if that happened or what the outcome was.
I've done this in the past regarding this issue, and the outcome was a no go. Couldn't hurt to do again, but I don't believe anything has changed. I am not a lawyer, and know legal would prefer engineers don't do too much lawyer cosplay. So I'll avoid posting my understanding here in the interest of not muddying the waters.
|
|
||
Comment 8•3 years ago
|
||
If Panasonic would issue a patent license for Decoded Picture Buffer, Mozilla might be able to do bug 1737116 before 2023-07-07 (h264 hardware decoding support without depending on full ffmpeg).
|
|
||
Comment 9•3 years ago
|
||
Please ask Mozilla legal if they could consider this.
|
|
||
Comment 10•3 years ago
|
||
Most decoded picture buffer patents expire 2023, but not all: bug 1737116 comment 13
2020-04-11 was the last time I tried to reproduce this bug (bug 1628428 comment 4).
It was closed as duplicate of bug 1628407, which added ffmpeg-full, but which has been backed out.
bug 1639531 then added org.freedesktop.Platform.ffmpeg-full as a recommendation.
Reconfirmed on Gnome Xwayland, Debian Testing:
h264 video stutters by default, it can be fixed with $ flatpak install flathub org.freedesktop.Platform.ffmpeg-full.
$ sudo apt install flatpak
$ sudo flatpak remote-add --if-not-exists flathub https://flathub.org/repo/flathub.flatpakrepo
$ flatpak list
$ flatpak install flathub org.mozilla.firefox
Looking for matches…
Required runtime for org.mozilla.firefox/x86_64/stable (runtime/org.freedesktop.Platform/x86_64/21.08) found in remote flathub
Do you want to install it? [Y/n]: y
org.mozilla.firefox permissions:
ipc network pcsc pulseaudio wayland
x11 devices file access [1] dbus access [2] bus ownership [3]
system dbus access [4]
[1] xdg-download
[2] org.a11y.Bus, org.freedesktop.FileManager1, org.freedesktop.Notifications, org.freedesktop.ScreenSaver, org.gnome.SessionManager,
org.gtk.vfs.*
[3] org.mozilla.firefox.*, org.mpris.MediaPlayer2.firefox.*
[4] org.freedesktop.NetworkManager
KENNUNG Zweig Op Remote Download
1. [✓] org.freedesktop.Platform.GL.default 21.08 i flathub 130,9 MB / 131,2 MB
2. [✓] org.freedesktop.Platform.Locale 21.08 i flathub 23,7 MB / 325,8 MB
3. [✓] org.freedesktop.Platform.VAAPI.Intel 21.08 i flathub 11,7 MB / 11,7 MB
4. [✓] org.freedesktop.Platform.openh264 2.0 i flathub 1,5 MB / 1,5 MB
5. [✓] org.gtk.Gtk3theme.Adwaita-dark 3.22 i flathub 2,4 kB / 1,5 kB
6. [✓] org.freedesktop.Platform 21.08 i flathub 153,4 MB / 199,6 MB
7. [✓] org.mozilla.firefox.Locale stable i flathub 507,2 kB / 46,3 MB
8. [✓] org.mozilla.firefox stable i flathub 84,5 MB / 86,1 MB
Installation complete.
$ flatpak list
Name Application ID Version Zweig Installation
Freedesktop Platform org.freedesktop.Platform 21.08.9 21.08 system
Mesa org.freedesktop.Platform.GL.default 21.3.3 21.08 system
Intel org.freedesktop.Platform.VAAPI.Intel 21.08 system
openh264 org.freedesktop.Platform.openh264 2.1.0 2.0 system
Adwaita dark GTK theme org.gtk.Gtk3theme.Adwaita-dark 3.22 system
Firefox org.mozilla.firefox 96.0.3 stable system
$ flatpak info org.mozilla.firefox
Firefox - Fast, Private & Safe Web Browser
Kennung: org.mozilla.firefox
Ref: app/org.mozilla.firefox/x86_64/stable
Architektur: x86_64
Zweig: stable
Version: 96.0.3
License: MPL-2.0
Ursprung: flathub
Sammlung: org.flathub.Stable
Installation: system
Installiert: 240,1 MB
Laufzeitumgebung: org.freedesktop.Platform/x86_64/21.08
Sdk: org.freedesktop.Sdk/x86_64/21.08
Commit: e80dda5e17d05dd18fe0ccb18d179f9ac525fc63e61ee2beef9991a0a5ef47d9
Parent: c19d597a8100cca427e672a39313b2a469a9b14c311e3d8a560ee8a54e5f08ea
Subject: Export org.mozilla.firefox
Date: 2022-01-27 14:46:23 +0000
$ flatpak info org.mozilla.firefox -m
[Application]
name=org.mozilla.firefox
runtime=org.freedesktop.Platform/x86_64/21.08
sdk=org.freedesktop.Sdk/x86_64/21.08
base=app/org.mozilla.firefox.BaseApp/x86_64/21.08
command=firefox
required-flatpak=0.11.1
[Extension org.mozilla.firefox.Locale]
directory=share/runtime/langpack
autodelete=true
locale-subset=true
[Extension org.freedesktop.Platform.ffmpeg-full]
directory=lib/ffmpeg
add-ld-path=.
no-autodownload=true
version=21.08
[Context]
shared=network;ipc;
sockets=x11;wayland;pulseaudio;pcsc;
devices=all;
filesystems=xdg-download;
persistent=.mozilla;
[Session Bus Policy]
org.freedesktop.Notifications=talk
org.freedesktop.FileManager1=talk
org.mozilla.firefox.*=own
org.mpris.MediaPlayer2.firefox.*=own
org.a11y.Bus=talk
org.freedesktop.ScreenSaver=talk
org.gtk.vfs.*=talk
org.gnome.SessionManager=talk
[System Bus Policy]
org.freedesktop.NetworkManager=talk
$ flatpak run org.mozilla.firefox
$ flatpak run org.mozilla.firefox https://bug1619882.bmoattachments.org/attachment.cgi?id=9149605
--> video stutters
$ flatpak install flathub org.freedesktop.Platform.ffmpeg-full
Looking for matches…
Similar refs found for ‘org.freedesktop.Platform.ffmpeg-full]’ in remote ‘flathub’ (system):
1) runtime/org.freedesktop.Platform.ffmpeg-full/x86_64/19.08
2) runtime/org.freedesktop.Platform.ffmpeg-full/x86_64/20.08
3) runtime/org.freedesktop.Platform.ffmpeg-full/x86_64/21.08
Which do you want to use (0 to abort)? [0-3]: 3
KENNUNG Zweig Op Remote Download
1. [✓] org.freedesktop.Platform.ffmpeg-full 21.08 i flathub 4,1 MB / 4,2 MB
Installation complete.
$ flatpak run org.mozilla.firefox https://bug1619882.bmoattachments.org/attachment.cgi?id=9149605
-> video does not stutter
$ flatpak uninstall org.freedesktop.Platform.ffmpeg-full
KENNUNG Zweig Op
1. [-] org.freedesktop.Platform.ffmpeg-full 21.08 r
Uninstall complete.
$ flatpak run org.mozilla.firefox https://bug1619882.bmoattachments.org/attachment.cgi?id=9149605
--> video stutters again
|
||
Comment 11•3 years ago
|
||
If there's need for legal clarification, I believe https://bugzilla.mozilla.org/enter_bug.cgi?product=Legal can be helpful.
Decodement by 'http://codecs.fedoraproject.org/openh264/36/x86_64/os/Packages/m/mozilla-openh264-2.1.1-3.fc36.x86_64.rpm' during utilisation of 'http://fedora.mirror.liteserver.nl/linux/development/rawhide/Everything/x86_64/os/Packages/f/firefox-96.0-1.fc36.x86_64.rpm' is slow, so is this problem solely relevant to flatpak?
|
||
Comment 14•3 years ago
|
||
(In reply to BEEDELL ROKE JULIAN LOCKHART from comment #12)
Decodement by 'http://codecs.fedoraproject.org/openh264/36/x86_64/os/Packages/m/mozilla-openh264-2.1.1-3.fc36.x86_64.rpm' during utilisation of 'http://fedora.mirror.liteserver.nl/linux/development/rawhide/Everything/x86_64/os/Packages/f/firefox-96.0-1.fc36.x86_64.rpm' is slow, so is this problem solely relevant to flatpak?
It's not, it's about missing ffmpeg libraries with patented algorithms. Fedora also has the similar problem as long as it cannot ship ffmpeg-libs because of legal reasons. The problem is that flatpak does not load system libraries, so even if you have ffmpeg-libs rpm from rpmfusion installed the firefox flatpak cannot use it. It needs org.freedesktop.Platform.ffmpeg-full extensions which puts the ffmpeg libraries to the org.freedesktop.Platform which firefox flatpak can use.
|
||
Updated•2 years ago
|
|
||
Comment 15•2 years ago
|
||
Could openSUSEs approach be adapted for the packaging of firefox´s OpenH264 package?
Description
•