(In reply to Florian Bender from bug 760140 comment #30) > Thanks, great. Do you know if GStreamer is using AV Foundation on Mac, or if > there is a translation layer for GStreamer i/f to AV Foundation backend > available? If so, it wouldn't even be necessary to do all the Mac-only work > (where AV Foundation is intended to be used, see Bug 801521). Otherwise we > should investigate if AV Foundation provides any benefits GStreamer cannot > (e. g. latency, leveraging the hw at its fullest and best?). We may > eventually be able to combine all the backends of the different platforms > and use a single interface for interacting with them (which means less > maintenance work and easier and better interop). (In reply to Alessandro Decina from bug 760140 comment #31) > I don't know what's the state of #801521, I'd be all for using GStreamer on > Mac. On Mac gst uses a lower level API called VideoToolBox, which is what is > used by AVFoundation under the hood. At the time the gst plugin was written, > AVFoundation didn't expose all the necessary interfaces to implement a gst > plugin on top of it. Since last year or so it does, and I have some > uncommitted code that uses it that I could get in shape and merge. (In reply to Chris Pearce (:cpearce) from bug 760140 comment #32) > Bugger me. Maybe we can ship gstreamer-core with Firefox on Mac with the > GStreamer plugin that uses VideoToolbox. Then we don't need to write another > AVFoundation backend.

For the very little that it counts, using GStreamer would be optimal for TenFourFox, since the code already exists in at least of its derivatives (Tobias' work in bug 760140 is based on his original proof of concept in AuroraFox). In terms of portability, it seems like it would be the most adaptable solution, and the VT plugin would advance performance further on 10.6+ (especially good since AVF is 10.7+ only).

Would importing gstreamer in m-c be a requirement for this? Ideally if possible I would make the build pull the binaries provided by the gstreamer sdk (there are binaries for mac/windows/linux), and only import the VTB/AVFoundation plugin in m-c.

I think we'd ideally like the gstreamer code in m-c, so that we know exactly what version we're shipping and our users are running.

Seems to me that we want to do this. GStreamer is already used on Linux platforms to allow decoding of H.264, MP3 and AAC (is it? At least there is some code already available!). GStreamer can also be enabled on Mac OS (in fact, the Linux code was at least partly developed and tested on Mac OS). GStreamer uses the same APIs as AV Foundation (which was only introduced in 10.7 whereas Firefox requires at least Mac OS X 10.6 – GStreamer can handle that!). According to comment 3 by Chris Pearce (:cpearce), GStreamer should live in m-c (instead of linking to pre-built libraries). The only downside seems to be a bigger codebase than minimally necessary, but I think this is a negligible sacrifice when compared to the required developer time. We can still replace the GStreamer backend with an AV Foundation backend in future versions. Any objections? If not, please change this bug's status to NEW (or ASSIGNED if anybody volunteers right now?) and mark Bug 801521 as a duplicate of this bug (I can't) – thanks!

I've been investigating this approach for the past few days. Looks like the best approach on Mac if we can get the size of the binaries within reasonable limits.

I can definitely give this a shot if we agree it's something worth trying. Is there a tool to help with the autotools => mozbuild conversion or is it all done manually?

Normally, we treat the build systems of external projects as black boxes - no moz.build conversion necessary. You just reference the external project as an EXTERNAL_DIRS or add_tier_directory('foo', static=True) and everything just works. If the project has a configure script, we'll need to hook that up in configure.in. If we need to link the resulting object files into libxul, we'll also have some additional steps to report those object files with our build system.

We actually have examples of both in the tree, so it's not *quite* that clear-cut. For example, basically everything under media/ is a third-party library: http://mxr.mozilla.org/mozilla-central/source/media/ but they're all built using moz.build+Makefiles.

This may be a stupid question, but knowing Firefox 21 is supporting MP3 through gstreamer on Linux, I took the mozilla-release hg repository, built Firefox WITH gstreamer support on OS X (using gstreamer-0.10 from MacPorts took one little change to one header file due to Firefox compiling with C++11 support and gstreamer-0.10 being broken in this very file), but I still can't play anything on i.e. soundcloud or my own HTML5 audio player with Flash being disabled. Did I miss anything in particular?

http://gstreamer.freedesktop.org/features/ https://developer.apple.com/library/prerelease/mac/releasenotes/MacOSX/WhatsNewInOSX/Articles/MacOSX10_9.html#//apple_ref/doc/uid/TP40013207-SW3 gstreamer uses the quicktime framework which is now deprecated in os x 10.9 mavericks, I'd strongly discourage adopting this approach considering that it won't work for long. Unless you want to port gstreamer to avfoundation.

The text on gstreamer.freedesktop.org is probably very old. GStreamer doesn't use QuickTime on osx.

(In reply to Cam from comment #10) > http://gstreamer.freedesktop.org/features/ > > https://developer.apple.com/library/prerelease/mac/releasenotes/MacOSX/ > WhatsNewInOSX/Articles/MacOSX10_9.html#//apple_ref/doc/uid/TP40013207-SW3 > > gstreamer uses the quicktime framework which is now deprecated in os x 10.9 > mavericks, I'd strongly discourage adopting this approach considering that > it won't work for long. Unless you want to port gstreamer to avfoundation. As Alessandro said, the text there is out of date. The applemedia plugin now uses VideoToolbox for decoding.

I've had a brief look at integrating gstreamer into the build, with some success. So far I've been able to get gstreamer core building and linking. As gstreamer is LGPL we can't statically link it, so it's built into its own .so and linked in later. Still needed to do is get applemedia and qtdemux plugins building, as well as glib. Also we'll want to have a static plugin registry. This is what I have at the moment, though. Just hoping to get some feedback to make sure I'm going in the right direction here. Patches to follow...

Import script for gstreamer sources. Here we `parse' (with sed) selected Makefile.am files from gstreamer for the bits we need, copy them over, and put together a Makefile.in and moz.build for each directory.

Extras we need to get gstreamer to build.

moz.build for media/libgstreamer and template Makefile.in for subdirs.

Build system stuff.

Trivial miscellaneous changes.

Auto-generated build files in media/libgstreamer/*/

Comment on attachment 789358 [details] [diff] [review] 851290-1-import-script.patch Review of attachment 789358 [details] [diff] [review]: ----------------------------------------------------------------- I don't care about the script used to import code as much as I care about the checked in build files. Since I am pretty busy and since you should be able to find someone else to review this patch, I'm going to ignore it for now. That being said, a lot of the shell scripts in the tree are being ported to Python, mainly for maintainability reasons. I'd encourage you to consider writing the import script in Python.

Comment on attachment 789361 [details] [diff] [review] 851290-3-makefiles.patch Review of attachment 789361 [details] [diff] [review]: ----------------------------------------------------------------- ::: media/libgstreamer/Makefile.in.template @@ +7,5 @@ > +srcdir = @srcdir@ > +VPATH = @srcdir@ > + > +GST_PATH = @top_srcdir@/media/libgstreamer > +GST_BUILD_PATH = @DEPTH@/media/libgstreamer What's this variable for? @@ +19,5 @@ > +LOCAL_DEFINES = -DHAVE_CONFIG_H -DGST_DISABLE_GST_DEBUG -UHAVE_ORC > +LOCAL_HEADERS = -include $(GST_PATH)/compat/gcc_visibility.h > +LOCAL_INCLUDES = -I$(GST_PATH)/compat > + > +LOCAL_CFLAGS = $(LOCAL_DEFINES) $(LOCAL_HEADERS) $(LOCAL_INCLUDES) The convention is UPPERCASE variables are reserved for actual build system variables and all other variables are for local use (this is enforced in moz.build files). It may surprise you, but the only LOCAL_* variable that actually does anything is LOCAL_INCLUDES. Everything else has no effect. Of course, you circumvent this by modifying CFLAGS and CXXFLAGS below, so all is well. But, the local variables do break convention, so this is wrong.

Comment on attachment 789362 [details] [diff] [review] 851290-4-build.patch Review of attachment 789362 [details] [diff] [review]: ----------------------------------------------------------------- I'm going to defer to Mike Hommey for configure.in bits. Otherwise this looks mostly good. One thing to keep in mind - and this is the major question I have before this all gets checked in - is whether we should compile gstreamer using our build system or upstream's. There are advantages to using our build system - we don't have to worry about integrating yet another 3rd party build system into ours. A downside is there might be build voodoo in gstreamer's build system we might be ignoring. This could affect stability (e.g. workarounds to known bugs). Before we use our own build system, we should probably audit gstreamer's build system for any quirks that need carried forward. Finally, I'd like to thank you for running this by the build peers before checkin. We've had fire drills in the past where groups wanted to land a new external project like this and expected a build config rubber stamp. Thank you for not doing that :)

Comment on attachment 789362 [details] [diff] [review] 851290-4-build.patch Review of attachment 789362 [details] [diff] [review]: ----------------------------------------------------------------- ::: configure.in @@ +5788,5 @@ > + else # Building our own GStreamer > + PKG_CHECK_MODULES(GSTREAMER, > + glib-2.0 gmodule-2.0 gobject-2.0, > + GSTREAMER_DEPS="$GSTREAMER_LIBS", > + AC_MSG_ERROR([glib-2.0 is needed to build gstreamer])) Is it okay to add this requirement on mac, or would we need to import glib too? And since you're running gstreamer's configure, why is that needed at all? @@ +5795,5 @@ > + GSTREAMER_LIBS='$(DEPTH)/media/libgstreamer/gstreamer/$(DLL_PREFIX)gstreamer$(DLL_SUFFIX) -lgstreamer'" $GSTREAMER_LIBS" > + GSTREAMER_LIBS='$(DEPTH)/media/libgstreamer/gstbase/$(DLL_PREFIX)gstbase$(DLL_SUFFIX) -lgstbase'" $GSTREAMER_LIBS" > + GSTREAMER_LIBS='$(DEPTH)/media/libgstreamer/gstvideo/$(DLL_PREFIX)gstvideo$(DLL_SUFFIX) -lgstvideo'" $GSTREAMER_LIBS" > + GSTREAMER_LIBS='$(DEPTH)/media/libgstreamer/gstapp/$(DLL_PREFIX)gstapp$(DLL_SUFFIX) -lgstapp'" $GSTREAMER_LIBS" > + GSTREAMER_LIBS='$(DEPTH)/media/libgstreamer/compat/$(DLL_PREFIX)gstcompat$(DLL_SUFFIX) -lgstcompat'" $GSTREAMER_LIBS" Looks like you should be using $(call EXPAND_LIBNAME_PATH), see e.g. MOZ_JPEG_LIBS. @@ +5803,2 @@ > AC_SUBST(GSTREAMER_CFLAGS) > +AC_SUBST(GSTREAMER_DEPS) is this really going to be used? @@ +5812,5 @@ > AC_DEFINE_UNQUOTED(GST_API_VERSION, "$GST_API_VERSION") > fi > > +if test -n "$MOZ_SYSTEM_GSTREAMER"; then > + AC_DEFINE(MOZ_SYSTEM_GSTREAMER) I'd rather not add an AC_DEFINE if it can be replaced by a: ifdef FOO DEFINES += -DFOO endif in a few Makefile.ins. @@ +9404,5 @@ > > +if test "$MOZ_GSTREAMER" -a -z "$MOZ_SYSTEM_GSTREAMER"; then > + if ! test -e media/libgstreamer; then > + mkdir -p media/libgstreamer > + fi Your toolkit.mozbuild change suggests media/libgstreamer will have a Makefile. If it does, at this point it already exists, so there's no need for this check. ::: toolkit/library/Makefile.in @@ +399,5 @@ > endif > > +ifdef MOZ_GSTREAMER > +ifndef MOZ_SYSTEM_GSTREAMER > +EXTRA_DSO_LDOPTS += $(GSTREAMER_LIBS) You have this for both MOZ_SYSTEM_GSTREAMER and not MOZ_SYSTEM_GSTREAMER case. Just do: ifdef MOZ_GSTREAMER EXTRA_DSO_LDOPTS += $(GSTREAMER_LIBS) endif

Hey guys, Is there a reasonable expectation of when we can expect this to be resolved?

I have been working on this for the past month or two and I'm ready to push it up for review. I'm waiting for https://bugzilla.mozilla.org/show_bug.cgi?id=806917 to land, then hopefully this will quickly follow.

We probably want to run some fuzz tests against GStreamer on MacOS in the near future.

I realize things may have changed and people are considering doing something else on Mac, but i'm still going to wrap this up since I spent a fair amount of time on it. So, WIP build with bundled gst 1.2.2. H264 and aac decoding should work on both 32 and 64 bits: http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/alessandro.d@gmail.com-3e8c4ce195b5/try-macosx64/

(In reply to Alessandro Decina from comment #28) > I realize things may have changed and people are considering doing something > else on Mac, but i'm still going to wrap this up since I spent a fair amount > of time on it. So, WIP build with bundled gst 1.2.2. H264 and aac decoding > should work on both 32 and 64 bits: > http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/alessandro.d@gmail. > com-3e8c4ce195b5/try-macosx64/ ...and mp3

(In reply to Alessandro Decina from comment #28) > I realize things may have changed and people are considering doing something > else on Mac, but i'm still going to wrap this up since I spent a fair amount > of time on it. So, WIP build with bundled gst 1.2.2. H264 and aac decoding > should work on both 32 and 64 bits: > http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/alessandro.d@gmail. > com-3e8c4ce195b5/try-macosx64/ It seems to work fine on youtube using the html5 player. The decoding capabilities seem to be detected fine. One question: Does gstreamer support hardware decoding?

(In reply to beingalink from comment #30) > (In reply to Alessandro Decina from comment #28) > > I realize things may have changed and people are considering doing something > > else on Mac, but i'm still going to wrap this up since I spent a fair amount > > of time on it. So, WIP build with bundled gst 1.2.2. H264 and aac decoding > > should work on both 32 and 64 bits: > > http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/alessandro.d@gmail. > > com-3e8c4ce195b5/try-macosx64/ > > It seems to work fine on youtube using the html5 player. The decoding > capabilities seem to be detected fine. One question: Does gstreamer support > hardware decoding? Thanks for testing! And yes, that build is doing h264 hw decoding.

I can confirm too, that this build works. I have Mac 10.9. YouTube and "normal" html5/h264-Videoplayer works. I think that you already know, but MSE & H.264 is not supported according to YouTube although I have actived MSE one about:config.

fwiw, I'm pushing this here: https://github.com/alessandrod/gecko-dev/commits/1.0-system

(In reply to patrickalbrecht1996 from comment #32) > I can confirm too, that this build works. > > I have Mac 10.9. > > YouTube and "normal" html5/h264-Videoplayer works. > I think that you already know, but MSE & H.264 is not supported according to > YouTube although I have actived MSE one about:config. Yup, I haven't looked into MSE at all (yet).

This is the compiled size. It can be easily taken down to 3MB by not compiling some stuff that i'm compiling but that it isn't used. alessandro@june:~/src/gecko-dev$ ./mach python /Users/alessandro/src/stuff/ldmap.py --prefix ../../media/ gstreamer --depth 3 /tmp/firefox-map ../../media/gstreamer 4.09MB ../../media/gstreamer/mozbuild 4.09MB ../../media/gstreamer/mozbuild/glib 614.41KB ../../media/gstreamer/mozbuild/gst-plugins-bad 629.57KB ../../media/gstreamer/mozbuild/gst-plugins-base 1.14MB ../../media/gstreamer/mozbuild/gst-plugins-good 379.84KB ../../media/gstreamer/mozbuild/gstreamer 1.10MB ../../media/gstreamer/mozbuild/orc 266.23KB

(In reply to Alessandro Decina from comment #34) > (In reply to patrickalbrecht1996 from comment #32) > > YouTube and "normal" html5/h264-Videoplayer works. > > I think that you already know, but MSE & H.264 is not supported according to > > YouTube although I have actived MSE one about:config. > > Yup, I haven't looked into MSE at all (yet). It's still under development, if there are GStreamer related changes I'll loop you in. The immediate reason that it shows as unsupported is most likely due to our AVC format whitelist, but even then I wouldn't expect things to work yet... coming soon!

I have experienced a hard system freeze when watching a YT video with that. Ouch.

(In reply to Mihai Moldovan from comment #37) > I have experienced a hard system freeze when watching a YT video with that. > Ouch. Thanks for testing! That's interesting, can you provide some more info? What OSX version? What video (link)? Can you reproduce it? Meanwhile, tests seem to be stable enough: https://tbpl.mozilla.org/?tree=Try&rev=76d9668aae62. There's an intermittent failure that i'm handling right now.

It seems MSE & H.264 doesn't work on Windows and Linux nightlies either. So this is not "GStreamer on Mac"-only specific issue.

(In reply to Alessandro Decina from comment #38) > Thanks for testing! That's interesting, can you provide some more info? What > OSX version? What video (link)? Can you reproduce it? Hardly. To be fair, I've had an system uptime (OS X 10.9) of about 3 weeks, the session contained more than 300 other tabs and I've been already watching other H254 videos at the time, so I wouldn't bet on what else came into the equation of a full system freeze. I tried loading up the monstrous session and watching the crashing video in question, only to see Firefox crashing again (albeit at a different time within the video.) This said, I can’t reproduce the problem with a clean session just watching the video as the first (or, well, second tab, as the first tab has been the recovery tab.) Also to be even more fair, I don’t want to post the full URL, as the video is basically an old TV advertisement I’ve been watching for fun. If you’d like to try to reproduce, here’s the video ID: ZTpXh33Mbeg

I did some quick testing on youtube, vimeo, and various h.264 test sites, and everything played back nicely on two iMacs using 10.8.5 and 10.9.1.

(In reply to Per Olofsson from comment #41) > I did some quick testing on youtube, vimeo, and various h.264 test sites, > and everything played back nicely on two iMacs using 10.8.5 and 10.9.1. awesome, thanks a lot for testing especially on 10.8.x. Testing on 10.7 and 10.6 would also be very appreciated!

Btw, how much does the OS X gstreamer integration depend on the gst-1.0 changes? I'm thinking about building Firefox 27 (which has, originally, only gst-0.10 support) with gstreamer support (i.e., backporting the changes from mozcentral to mozrelease.) Is attachment #6 [details] everything I'll need to apply to mozrelease for that matter?

(In reply to Mihai Moldovan from comment #43) > Btw, how much does the OS X gstreamer integration depend on the gst-1.0 > changes? It's 100% dependent on 1.0. The bulk of the patch is adding gstreamer 1.2.2 to m-c :) > > I'm thinking about building Firefox 27 (which has, originally, only gst-0.10 > support) with gstreamer support (i.e., backporting the changes from > mozcentral to mozrelease.) Is attachment #6 [details] everything I'll need > to apply to mozrelease for that matter? No. All the attachments on this bug are obsolete. All the code is here: https://github.com/alessandrod/gecko-dev/commits/1.0-system (the top 6 commits).

(In reply to Alessandro Decina from comment #28) > I realize things may have changed and people are considering doing something > else on Mac, but i'm still going to wrap this up since I spent a fair amount > of time on it. So, WIP build with bundled gst 1.2.2. H264 and aac decoding > should work on both 32 and 64 bits: > http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/alessandro.d@gmail. > com-3e8c4ce195b5/try-macosx64/ Is there a current build location for this fork of gecko-dev?

Comment on attachment 789358 [details] [diff] [review] 851290-1-import-script.patch (In reply to Alessandro Decina from comment #44) > No. All the attachments on this bug are obsolete. All the code is here: > https://github.com/alessandrod/gecko-dev/commits/1.0-system (the top 6 > commits).

Are you still on to this?

Change of plan. We don't intend to ship support for GStreamer on MacOSX for MP4 support. I outlined our plan for MacOSX MP4 support here: https://groups.google.com/forum/#!topic/mozilla.dev.media/oxY_kp1Rq4s

I first wanted to check with Alessandro before WONTFIXing it since he showed interest despite the change of plans. I guess if you really don't want this stuff to live in the tree when it's solely used, then WONTFIXing it anyway is better, of course.