> Mike's suggestion in this thread to build the SDK from the Firefox > build instead of the XULRunner build is righteous. Someone should do > that! ;-) That would be a releng matter, since our build system has everything necessary for that.

From the thread: RelEng is involved, but not completely on the hook. Before we can do anything, we need to know what extra commands we need to run as part of the Firefox build to make this happen. At the very least, some adjustments to the upload rules will need to happen, because they don't support multiple products at the moment. I'm not sure how we can run 'make package' for XULRunner, either, because the mozconfig used will have Firefox values set.... So, that's a long way of saying that we need a demonstrated build process before we can automate it.

From the thread, too: Building the sdk off Firefox should only be a matter of running make sdk in the objdir, and uploading the resulting dist/sdk/firefox*.sdk.tar.bz2. Though looking into it, we don't have a "make sdk" on Firefox, but "make -C $objdir/browser/installer make-sdk" works.

Excellent! I didn't know that was possible. We still need to figure out how to make the upload rules work, since they depend on things that --enable-application sets IIRC.

From a quick reading of toolkit/mozapps/installer/packager.mk, as long as you run make upload after make -C $objdir/browser/installer make-sdk, the sdk will be uploaded.

OK, it sounds like we just need to run "make-sdk" sometime prior to uploading, then?

Sounds like we should just add a "make sdk" target to Firefox for simplicity's sake.

Found in triage.

sounds like some build system work is required here first. anybody up for that?

Implements comment 6.

Can we go ahead generating the sdk from Firefox, now?

Do you want to use this ticket for other tasks, like moving install_app.py? Or would you prefer this as a tracking ticket, or something else?

Those should be separate tasks.

make sdk is not sufficient on MacOS. dist/firefox-sdk/bin contains Nightly.app, not a XUL.framework folder. Some more build work is needed.

(In reply to Alex Vincent [:WeirdAl] from comment #15) > make sdk is not sufficient on MacOS. dist/firefox-sdk/bin contains > Nightly.app, not a XUL.framework folder. Some more build work is needed. There is no need to wait for more build work to start building the sdk so that we can actually see the result of build changes. Although we may not want to create the sdk on all builds. Maybe nightlies only? But then it would be useful to be able to trigger on try, too. Opinions from releng?

(In reply to Mike Hommey [:glandium] from comment #16) > (In reply to Alex Vincent [:WeirdAl] from comment #15) > > make sdk is not sufficient on MacOS. dist/firefox-sdk/bin contains > > Nightly.app, not a XUL.framework folder. Some more build work is needed. > > There is no need to wait for more build work to start building the sdk so > that we can actually see the result of build changes. Although we may not > want to create the sdk on all builds. Maybe nightlies only? But then it > would be useful to be able to trigger on try, too. > > Opinions from releng? How much time does it take to generate the SDK after doing a Firefox build in the same directory? If we're talking about a few minutes, the best thing is just to do it everywhere IMO. If it takes more than a small amount of time we should probably just do it on nightlies (the same schedule we built XULRunner on right now). The latter makes it harder to test on try, though. One other point is that if we're doing these as part of existing Firefox builds we'll end up uploading the SDK to the same place as the build. Is that going to be okay? I presume that we explicitly don't want to upload it to anywhere under /pub/mozilla.org/xulrunner once we're done with this bug.

My experience says it's only a few minutes: * Creating omni.ja * Creating and populating a staging directory (dist/___-sdk) * Zipping up the staged files

(In reply to Alex Vincent [:WeirdAl] from comment #18) > My experience says it's only a few minutes: If this takes less than 5 minutes on RelEng machinery it's probably OK to do it on all builds, which would give us try-ability for free. > * Creating omni.ja This is a bit of an aside, but is there a reason we can't re-use the omni.ja that was created for Firefox?

We should not be creating a separate omni.ja. I'm mainly worried that we would be uploading the SDK for each build, which seems a bit silly. Can we only upload it from nightlies even if we build it all the time?

(In reply to Benjamin Smedberg [:bsmedberg] from comment #20) > Can we > only upload it from nightlies even if we build it all the time? That seems sensible. The easiest way to do this is to only add it to UPLOAD_EXTRA_FILES during a nightly build. MOZ_UPDATE_CHANNEL != default might be the easiest thing to look for...

what's our path forward here?

(In reply to Chris AtLee [:catlee] from comment #22) > what's our path forward here? The patch in bug 961936 is a superset of what's needed here. The only thing it's missing in that patch is a change to mach, so that mach supports the sdk target.

Is this a dupe of the now-fixed bug 1137000? If so, it sounds like we can kill XULRunner builds now....

It appears to be, and I see .sdk packages in latest-mozilla-central.

I'm not sure we can kill the xulrunner-built sdks just yet. See comment 15.

We can kill the XULRunner builds once we include the xulrunner-stub binary in the .sdk package. If my patch in bug 1105044 is valid, then we also needn't rely on a XUL.framework being present as it will be obsolete. Again, having the xulrunner-stub binary in there (perhaps in the lib/ folder of the sdk package?) is quite essential.

(In reply to Mike de Boer [:mikedeboer] from comment #27) > We can kill the XULRunner builds once we include the xulrunner-stub binary > in the .sdk package. > If my patch in bug 1105044 is valid, then we also needn't rely on a > XUL.framework being present as it will be obsolete. > > Again, having the xulrunner-stub binary in there (perhaps in the lib/ folder > of the sdk package?) is quite essential. That sounds more like a comment for bug 1147577. That is, xulrunner-stub is something that comes from xulrunner, not its SDK. The SDK is afaik only advertised to build binary components for Firefox. xulrunner-stub is a different thing, in that it allows to bootstrap xulrunner applications. Which is nice, but I think it should die in its current form, because it does all the libxul bootstrapping, like firefox does, but its code is rarely updated when firefox's is. For instance, it doesn't do DLL blocklist. I'm tempted to say the stub should be replaced with something that just executes 'firefox -app application.ini'

> I'm tempted to say the stub should be replaced with something that just executes 'firefox -app application.ini' Please don't. If someone is shipping a completely standalone application, they shouldn't have to ship the firefox executable and rely on an icon that launches Firefox with a parameter. They should have a named executable that launches the application. If you want to be really clever, you could make it so that if the EXE is named something other than firefox.exe, it automatically searches for application.ini.

(In reply to Mike Kaply [:mkaply] from comment #29) > > I'm tempted to say the stub should be replaced with something that just executes 'firefox -app application.ini' > > Please don't. If someone is shipping a completely standalone application, > they shouldn't have to ship the firefox executable and rely on an icon that > launches Firefox with a parameter. If they do that, they already have to hack the xulrunner-stub because it doesn't have the right icon for they application. Are there really completely standalone applications that just take a prebuilt xulrunner and do that instead of building gecko --with-application=my-app ?

(In reply to Mike Hommey [:glandium] from comment #30) > If they do that, they already have to hack the xulrunner-stub because it > doesn't have the right icon for they application. Are there really > completely standalone applications that just take a prebuilt xulrunner and > do that instead of building gecko --with-application=my-app ? No hack needed. Mozilla provides a tool for that. http://mxr.mozilla.org/mozilla-central/source/xulrunner/tools/redit/ And yes, people take XUL Runner and build applications. That's what it was designed for. There's no need to build gecko at all. You write a XUL application, package XUL Runner, rename the stub and you're good to go.

(In reply to Mike Kaply [:mkaply] from comment #31) > (In reply to Mike Hommey [:glandium] from comment #30) > > > If they do that, they already have to hack the xulrunner-stub because it > > doesn't have the right icon for they application. Are there really > > completely standalone applications that just take a prebuilt xulrunner and > > do that instead of building gecko --with-application=my-app ? > > No hack needed. Mozilla provides a tool for that. > > http://mxr.mozilla.org/mozilla-central/source/xulrunner/tools/redit/ That qualifies as a hack. And yet another tool that doesn't come in the Firefox SDK. > And yes, people take XUL Runner and build applications. That's what it was > designed for. There's no need to build gecko at all. You write a XUL > application, package XUL Runner, rename the stub and you're good to go. That was ten years ago. Do people still *actually* do this *now*? And if they do, do we actually need to support it? (as in, is it unreasonable to have them, for example, build the xulrunner-stub on their own (which could then have the right icon directly, and could even use the binary form of application.ini))

> That qualifies as a hack. And yet another tool that doesn't come in the Firefox SDK. It's not a hack. It's a tool. And it was in the XULRunner SDK. The fact that it's missing from the Firefox SDK is a bug. > That was ten years ago. Do people still *actually* do this *now*? Yes. I've done it for three clients this year. They're called site specific browsers. http://en.wikipedia.org/wiki/Site-specific_browser Why wouldn't you support it when it's so little effort? All the tools/technology are already there. They were there for the XUL Runner SDK.

Building XULRunner releases is not a Mozilla goal. Our goal is solely to provide the SDK so that people can continue building Firefox extensions against it (headers, link libraries, XPIDL). The only thing that would block turning off our XR builds would be if comment 15 means that the SDK packages on mac don't have these things. A stub or other parts of a XR binary are not blockers, and Mozilla won't be spending time on them.

I thought the point here was to allow people to use the Firefox SDK instead of XUL Runner. If that's the case, why wouldn't you provide the two missing pieces to make it easy to migrate (the stub and redit.exe)?

(In reply to Mike Kaply [:mkaply] from comment #35) > I thought the point here was to allow people to use the Firefox SDK instead > of XUL Runner. The point, since comment 0, was to build the SDK used to build binary components for addons from Firefox instead of Xulrunner.

(In reply to Mike Hommey [:glandium] from comment #36) > (In reply to Mike Kaply [:mkaply] from comment #35) > > I thought the point here was to allow people to use the Firefox SDK instead > > of XUL Runner. > > The point, since comment 0, was to build the SDK used to build binary > components for addons from Firefox instead of Xulrunner. To replace XUL Runner. And for it to replace XUL Runner, it has to have the same capability as XUL Runner. And that means the stub. You asked why someone wouldn't just build their app. If I did that, I'd have to set up build environments on all Mozilla platforms. With XUL Runner, I'm able to use the stubs on the various platforms and have XUL Runner applications with no hassle. I just don't understand why you guys hate this concept so much. Firefox is a great platform, not just a great browser.

Also, there are issues with the Windows taskbar if you try to use Firefox and Firefox launching a XUL Runner app. You need the stub for proper coexistence with Firefox.

(In reply to Benjamin Smedberg [:bsmedberg] from comment #34) > Building XULRunner releases is not a Mozilla goal. Our goal is solely to > provide the SDK so that people can continue building Firefox extensions > against it (headers, link libraries, XPIDL). The only thing that would block > turning off our XR builds would be if comment 15 means that the SDK packages > on mac don't have these things. > > A stub or other parts of a XR binary are not blockers, and Mozilla won't be > spending time on them. Do I understand correctly I will soon have no more workable and updated environments to build BlueGriffon and BlueGriffonEPUBEdition?

(In reply to Daniel Glazman (:glazou) from comment #39) > > A stub or other parts of a XR binary are not blockers, and Mozilla won't be > > spending time on them. > > Do I understand correctly I will soon have no more workable and updated > environments to build BlueGriffon and BlueGriffonEPUBEdition? Daniel, allow me to paraphrase Benjamin a bit: Mozilla won't be spending time on them, but that's been a reality for over two years, if not more. This doesn't mean that volunteers (including the spare time of fanatic Moz employees like myself) won't be. I've tried to contribute fixes to the XR binaries during my time here and I plan to commit to that for the foreseeable future. If Mozilla will start spending time in actively removing XR from the tree, I'm sure that will be publicly announced, hopefully along with a migration path. This includes the end result of the work done in this bug. I'd recommend to post to the dev-platform mailing list for a broader discussion than the scope of this bug allows if uncertainties and/ or question still remain.

(In reply to Mike de Boer [:mikedeboer] from comment #40) > I'd recommend to post to the dev-platform mailing list for a broader > discussion than the scope of this bug allows if uncertainties and/ or > question still remain. I will. Now that I've done the uplift to mozilla-central, BlueGriffonEPUBEdition will benefit from major recent CSS stuff. The work done by Simon Montagu on writing modes for instance is a major element for the Asian market. Japanese people are already excited about an EPUB editor being able to edit vertical Japanese. I will also add filters and all the cool new kids on the block to BlueGriffon, that will become, again, the most modern web editor on the market, based on Gecko. With millions of daily users around the world... In short, still an important part of the mozilla ecosystem. So your "spare time of fanatic employee" is more than welcome and please trust me on that, we never met but I'll happily invite for dinner when we meet. A zillion thanks.

(In reply to Mike Hommey [:glandium] from comment #32) > > And yes, people take XUL Runner and build applications... > > That was ten years ago. Do people still *actually* do this *now*? We use XUL Runner frequently in two scenarios: 1) Many of our enterprise customers simply won't install Firefox, usually because of the difficulty of locking it down sufficiently. Instead we supply a XUL Runner based application that connects directly to our application on their intranet web server and doesn't allow general browsing. 2) We make frequent use of XUL Runner internally to allow our front-end developers to easily use their HTML/XUL/CSS/JS skills to create stand-alone tools for a variety of purposes. Because these are only used internally, the icon issue doesn't bother us. > ...is it unreasonable to have them, for example, build the xulrunner-stub on their own... If I understand correctly the xulrunner-stub would be required in conjunction with Firefox. That would make it a non-starter for our enterprise customers. For our internal use we don't currently need to know anything about building Firefox or the stub. We can just grab a XUL Runner build for the target platform, hack in some quick client code, and we've got a ready-to-run "application", regardless of whether they've currently got Firefox on their system or not. Having to build the stub (and ship Firefox?) adds extra steps that would likely hinder this approach. I'm more concerned about (1) than (2). Until Firefox is more "enterprise friendly", and accepted by IT departments, XUL Runner provides a valuable mechanism for meeting corporate requirements. I understand that enterprise users aren't the main focus for Mozilla, but hidden behind corporate firewalls are thousands of end users relying on your technology every day. Please don't leave them, and the companies that supply them, high and dry. At the very least, please formalise any decision to drop XUL Runner, with definite timelines and further details, so that we can at least start to investigate and plan possible alternatives.

(In reply to Mark C from comment #42) > At the very least, please formalise any decision to drop XUL Runner, with > definite timelines and further details, so that we can at least start to > investigate and plan possible alternatives. I have been asking for that for ages, and never got any official answer so good luck.

Here's the bug onthe interaction issues on the Windows taskbar when using Firefox as a XULRunner launcher: https://bugzilla.mozilla.org/show_bug.cgi?id=942938 It's one of the reasons you need a stub.

mkaply asked me to weigh in. My work to date has been more experimental than actual product, so take my thoughts with a grain of salt. In my pet project work, my platform situation is quite a mess. This has to do with packaging of Firefox, more than anything else. On Linux, I can download a Firefox executable .tar.bz2, unpack it, and run Firefox using -app path/to/application.ini. Note that I have to provide an absolute path, which is mildly annoying, but I can get my work done no problem. On Windows, I have to download the XULRunner SDK because Firefox is only available on ftp.m.o as an installer. Windows Firefox doesn't have a tarball or zip file to download. On Macintosh, I have to manually build Firefox. I couldn't use XULRunner on Mac to launch my application reliably, and Firefox is only available as a DMG. On the plus side, I no longer have to move everything inside a Foo.app/Contents/Resources/ folder anymore...

This patch turns it on for CI builds as well as nightlies and releases. I'm still not sure if that's what we want. We need it for releases, clearly, not sure about nightlies or CI.

Using Firefox to run XULRunner apps on Windows is broke. See bug 942938. It's also painful to have a custom Mac app that launches Firefox this way. Please provide a stub executable (the same way that WebApps use a stub). It's the only way to truly use the Firefox as a replacement for XULRunner.

Comment on attachment 8632785 [details] [diff] [review] turn on sdk generation as part of Firefox builds Review of attachment 8632785 [details] [diff] [review]: ----------------------------------------------------------------- ::: build/mozconfig.automation @@ +16,5 @@ > mk_add_options "export MOZ_AUTOMATION_INSTALLER=${MOZ_AUTOMATION_INSTALLER-0}" > mk_add_options "export MOZ_AUTOMATION_UPDATE_PACKAGING=${MOZ_AUTOMATION_UPDATE_PACKAGING-0}" > mk_add_options "export MOZ_AUTOMATION_UPLOAD=${MOZ_AUTOMATION_UPLOAD-1}" > mk_add_options "export MOZ_AUTOMATION_UPLOAD_SYMBOLS=${MOZ_AUTOMATION_UPLOAD_SYMBOLS-0}" > +mk_add_options "export MOZ_AUTOMATION_SDK=${MOZ_AUTOMATION_SDK-1}" That's already set in build/macosx/mozconfig.common, build/mozconfig.win-common and build/unix/mozconfig.linux.

(In reply to Mike Hommey [:glandium] from comment #48) > Comment on attachment 8632785 [details] [diff] [review] > turn on sdk generation as part of Firefox builds > > Review of attachment 8632785 [details] [diff] [review]: > ----------------------------------------------------------------- > > ::: build/mozconfig.automation > @@ +16,5 @@ > > mk_add_options "export MOZ_AUTOMATION_INSTALLER=${MOZ_AUTOMATION_INSTALLER-0}" > > mk_add_options "export MOZ_AUTOMATION_UPDATE_PACKAGING=${MOZ_AUTOMATION_UPDATE_PACKAGING-0}" > > mk_add_options "export MOZ_AUTOMATION_UPLOAD=${MOZ_AUTOMATION_UPLOAD-1}" > > mk_add_options "export MOZ_AUTOMATION_UPLOAD_SYMBOLS=${MOZ_AUTOMATION_UPLOAD_SYMBOLS-0}" > > +mk_add_options "export MOZ_AUTOMATION_SDK=${MOZ_AUTOMATION_SDK-1}" > > That's already set in build/macosx/mozconfig.common, > build/mozconfig.win-common and build/unix/mozconfig.linux. Oh, I'm a dope! I see that they're already being uploaded as part of nightlies: http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2015-07-14-03-02-06-mozilla-central/. Is there anything left to do here, in that case? I'd like to help push this along however I can.

Enabling on beta/release, I guess, whatever that means (I /guess/ the SDKs are built, but are not on the ftp). And, obviously, there's all the stuff in the dependency graph (like the dual-sdk on mac ; dependencies might be in the wrong direction btw)

(In reply to Mike Hommey [:glandium] from comment #50) > Enabling on beta/release, I guess, whatever that means (I /guess/ the SDKs > are built, but are not on the ftp). They might not be built because we use different mozconfigs there. I'll look into it. > And, obviously, there's all the stuff in > the dependency graph (like the dual-sdk on mac ; dependencies might be in > the wrong direction btw) I _think_ there's nothing to do in the deps, other than bug 1165512. bug 961936 is not a blocker AFAIK (at the very least, bsmedberg has said on multiple occasions that the stub is not going to block us), and the other two are things that are truly blocked by this (new release automation, killing xulrunner builds).

There's no common mozconfigs for release-style builds, so putting them in each platform's seems like the right thing. When release automation comes around we'll have to reconsider this - that project is going to have us shipping CI binaries, so I suspect we'll need to generate SDKs for all CI builds on beta/release. We can make that change later though...

Comment on attachment 8636165 [details] [diff] [review] build sdk as part of beta + release Review of attachment 8636165 [details] [diff] [review]: ----------------------------------------------------------------- ::: browser/config/mozconfigs/linux32/beta @@ +2,5 @@ > > ac_add_options --enable-official-branding > > mk_add_options MOZ_PGO=1 > +mk_add_options "export MOZ_AUTOMATION_SDK=1" This should be MOZ_AUTOMATION_SDK=${MOZ_AUTOMATION_SDK-1} put before any include. That said, if there's an environment variable like there is IS_NIGHTLY, that could be used in build/macosx/mozconfig.common, build/mozconfig.win-common and build/unix/mozconfig.linux instead, that would be even better.

(In reply to Mike Hommey [:glandium] from comment #53) > Comment on attachment 8636165 [details] [diff] [review] > build sdk as part of beta + release > > Review of attachment 8636165 [details] [diff] [review]: > ----------------------------------------------------------------- > > ::: browser/config/mozconfigs/linux32/beta > @@ +2,5 @@ > > > > ac_add_options --enable-official-branding > > > > mk_add_options MOZ_PGO=1 > > +mk_add_options "export MOZ_AUTOMATION_SDK=1" > > This should be MOZ_AUTOMATION_SDK=${MOZ_AUTOMATION_SDK-1} put before any > include. That said, if there's an environment variable like there is > IS_NIGHTLY, that could be used in build/macosx/mozconfig.common, > build/mozconfig.win-common and build/unix/mozconfig.linux instead, that > would be even better. There's some variables that are only set for release by happenstance (eg, MOZ_PKG_PRETTYNAMES), but they don't seem like the right fit here. This will be fairly short lived anyways, per my previous comments about release promotion

Comment on attachment 8638557 [details] [diff] [review] better way of enabling sdk generation for betas and releases Review of attachment 8638557 [details] [diff] [review]: ----------------------------------------------------------------- meh

Comment on attachment 8638557 [details] [diff] [review] better way of enabling sdk generation for betas and releases https://hg.mozilla.org/integration/mozilla-inbound/rev/6be6deca7831

sorry had to back this out for failures like https://treeherder.mozilla.org/logviewer.html#?job_id=12251159&repo=mozilla-inbound

(In reply to Carsten Book [:Tomcat] from comment #58) > sorry had to back this out for failures like > https://treeherder.mozilla.org/logviewer.html#?job_id=12251159&repo=mozilla- > inbound D'oh. Should be an easy fix though. Sorry about the bustage.

Backout: https://hg.mozilla.org/integration/mozilla-inbound/rev/23d1a9b1df55

I ran these checks again locally to verify the failure + the fix. I also adjusted the export to use mk_add_options, which I think you asked for last time, but I was too dense to realize it.

(In reply to Ben Hearsum [:bhearsum] from comment #61) > Created attachment 8640584 [details] [diff] [review] > add sdk var to whitelist; fix initial export > > I ran these checks again locally to verify the failure + the fix. > > I also adjusted the export to use mk_add_options, which I think you asked > for last time, but I was too dense to realize it. I see no mk_add_options in the patch, and I'm not sure what you're talking about.

Comment on attachment 8640584 [details] [diff] [review] add sdk var to whitelist; fix initial export (In reply to Mike Hommey [:glandium] from comment #62) > (In reply to Ben Hearsum [:bhearsum] from comment #61) > > Created attachment 8640584 [details] [diff] [review] > > add sdk var to whitelist; fix initial export > > > > I ran these checks again locally to verify the failure + the fix. > > > > I also adjusted the export to use mk_add_options, which I think you asked > > for last time, but I was too dense to realize it. > > I see no mk_add_options in the patch, and I'm not sure what you're talking > about. Forgot to generate a new patch, heh. I landed this one, but can update with https://github.com/mozilla/gecko-dev/compare/master...bhearsum:beta-release-sdk if that's what you had intended.

No, it's fine as it is.

I think we're all done here now.

Unfortunately the SDK is built only for aurora and mozilla-central nightlies currently. I have opened bug #1233005 in order to suggest making it available for stable and beta releases too.