+++ This bug was initially created as a clone of Bug #590682 +++ Per bug 590682, 70% of the top 30 addons have a protocol handler of some sort. There is currently no pathway for these addons to migrate to WebExtensions, at least not based on the MDN docs as of this writing. There needs to be a way to establish the protocol handler, and for those extensions that implement channels or use sockets, a means to pass network data through the addon to the browser.

Could somebody clarify that 70% and actually look into if those protocol handlers are required or if the same thing could be solved differently. I wanted to check that, so I downloaded the 20 most popular addons from amo and only Greasemonkey seems to implement an nsIProtocolHandler.

We'll discuss this, but I should warn you at the outset that this will almost certainly not happen.

I would very much like to know what sort of protocols are being implemented in addons here. If nothing else, whether they only need navigation to work or general fetches is relevant for some standards work we're doing...

(In reply to Cameron Kaiser [:spectre] from comment #0) > There is currently no pathway for these addons to migrate to > WebExtensions, at least not based on the MDN docs as of this writing. That's certainly not true. Webextensions capabilities like web_accessible_resources (https://developer.mozilla.org/en-US/Add-ons/WebExtensions/manifest.json/web_accessible_resources) cover some (hopefully many or most) uses of protocol handlers. I'm sure there are add-ons that do other things that aren't currently possible in webextensions but I suspect this among the top uses and it is covered.

(In reply to Boris Zbarsky [:bz] from comment #3) > I would very much like to know what sort of protocols are being implemented > in addons here. If nothing else, whether they only need navigation to work > or general fetches is relevant for some standards work we're doing... Besides Chatzilla and IRC, there is also (disclaimer: this is an addon I maintain) OverbiteFF and Gopher, PCMAN BBS and Telnet, and FireFTP and FTP. This was just a cursory initial search of AMO, along with https://addons.mozilla.org/en-US/firefox/search/?q=protocol&appver=45.0&platform=mac(In reply to Andrew Swan [:aswan] from comment #4) (In reply to Andrew Swan [:aswan] from comment #4) > > There is currently no pathway for these addons to migrate to > > WebExtensions, at least not based on the MDN docs as of this writing. > > That's certainly not true. Webextensions capabilities like > web_accessible_resources > (https://developer.mozilla.org/en-US/Add-ons/WebExtensions/manifest.json/ > web_accessible_resources) cover some (hopefully many or most) uses of > protocol handlers. > > I'm sure there are add-ons that do other things that aren't currently > possible in webextensions but I suspect this among the top uses and it is > covered. Given that chrome:// and resource:// URLs already exist and do what it appears this feature does, I submit the reason that addons are using actual protocol handlers purely for internal resources is that the use case is not, in fact, covered.

(And more to the point, it doesn't cover the case of those addons that actually do implement _network_ protocols instead of using a protocol handler as essentially an alias.)

OK, so the irc case is basically a navigation case. So is telnet. ftp can probably be more or less navigation. Gopher I expect needs fetch hooks, right? Put another way, there are actually two separate problems that are both solved by protocol handlers right now but don't have to be solved via the same mechanism in general: 1) Some way to hook navigation to a URL that allows the hooker to respond with some arbitrary HTML/whatever content, running with either system permissions or not. 2) Some way to hook subresource (image, script, etc) fetches from a url. Generally speaking, a solution for #2 would also solve #1, but not vice versa, right?

Gopher would need a fetch hook. Right now it goes out, gets either a menu or a resource, and either formats the menu as HTML (and returns that) or gives the resource back as raw data. That would probably work also for FTP. So I guess that would be your option 1, but I'm not sure what option 2 means.

Option 2 is what you need to make <img src="gopher:whatever"> work (and similar for <script>, <video>, etc). I think gopher needs that, but none of irc/telnet/ftp do. For those it's enough to make <a href="irc:whatever"> and maybe <iframe src="irc:whatever"> work, so for those hooking navigation but not fetch is enough.

I get it now. Yes, option 2 would be the most seamless approach (it's more or less how I have it implemented now and those things all work), but if option 1 were the only choice it would still be substantially better than nothing, of course.

This is the list of addons from the top 30 that do implement their own nsIProtocolHandlers

Update: we threw the whole necko team into looking at the top 30 addons list in comment 11, both to test for e10s bustage and examine how they use nsIProtocolHandler. The status is actually much better than we were expecting. We checked 27 of them (3 weren't on AMO) and all of them actually are either already working under e10s or have plans to work under e10s (or are abandoned and already don't work on nightly with or without e10s turned on). FireFTP, for instance, does all its work in the Chrome/parent process, so even though it implements a full protocol stack on top of sockets, it still works. There are 16 more addons with >10K users which also use nsIProtocolHandler, and we'll be looking at them as well in the next few days. Beyond that, for reference, there are 32 addons with between 1K and 10k users, and 41 with less than 1K users. We're not planning to look at all of those. There are clearly some addons that will break: overbite (gopher://) is the one we know about for sure. But I think at this point the addons bustage is low enough that we can conclusively say that we don't need to block e10s on providing a webextension API. (Of course we'd like to have one as soon as possible). As far as what a webextension API should look like, we might as well make it compatible with the Electron API for protocol handlers: http://electron.atom.io/docs/api/protocol/ I'm not 100% clear on whether that API is useful for the Overbite case. It looks like if we also provide some sort of socket access, the 'buffer' version of the API might work, assuming you can call the 'callback()' function asynchronously--I'm not sure from the docs. Of course we'd need to think carefully about how to provide a socket API to addons.

(In reply to Jason Duell [:jduell] (needinfo me) from comment #12) > FireFTP, for instance, does all its work in the Chrome/parent process, > so even though it implements a full protocol stack on top of sockets, it > still works. Is there a way to force this to happen with a more generic approach? It sounds like the only reason it happens that way is because it opens a specific XUL chrome interface (like Chatzilla), and that does all the socket work. > As far as what a webextension API should look like, we might as well make it > compatible with the Electron API for protocol handlers: > > http://electron.atom.io/docs/api/protocol/ > > I'm not 100% clear on whether that API is useful for the Overbite case. It > looks like if we also provide some sort of socket access, the 'buffer' > version of the API might work, assuming you can call the 'callback()' > function asynchronously--I'm not sure from the docs. Of course we'd need to > think carefully about how to provide a socket API to addons. Actually, I think that general idea could work (looking at |registerBufferProtocol|). There would need to be a socket API, but this idea could conceivably do its socket work, create a Buffer with the response and set the equivalent MIME type based on the Gopher item type, and then hand that to the callback once the socket transaction was complete. This wouldn't work for something more interactive, and Gopher is a very trivial protocol, but it would at least cover this use case.

> > FireFTP, for instance, does all its work in the Chrome/parent process, so > > even though it implements a full protocol stack on top of sockets, it still > > works. > > Is there a way to force this to happen with a more generic approach? It sounds > like the only reason it happens that way is because it opens a specific XUL > chrome interface (like Chatzilla), and that does all the socket work. :billm, is there any straighforward way to migrate content from a child process to the parent? I.e. could we recognize that a particular protocol (like overbite's gopher://) needs to run in the parent and automagically have any top-level navigation to such a URI switch to use the parent process? (we don't need to support loading gopher:// subresources within an http:// page). I'm not suggesting we implement it for gopher if it doesn't exist already.

:bz might know the answer here too (see comment 13 onwards).

The code in E10SUtils.jsm determines which process a URL should run in. It's kind of ad-hoc right now. Bug 1095484 tried to add a more general mechanism, but it got bogged down somehow. I don't remember the details. The bug has more information. In general, though, we do have a mechanism to switch which process we load a top-level document URL in.

(In reply to Bill McCloskey (:billm) from comment #16) > The code in E10SUtils.jsm determines which process a URL should run in. It's > kind of ad-hoc right now. Bug 1095484 tried to add a more general mechanism, > but it got bogged down somehow. I don't remember the details. The bug has > more information. > > In general, though, we do have a mechanism to switch which process we load a > top-level document URL in. What about reversing that bug so that an add-on would have to request to load in the parent? That should require a much smaller patch and should be simple for add-ons to adopt (if I read it right, I could simply add the "must run in parent" flag and that would be enough). If this seems worth exploring, I can follow up in that bug and leave this for the WebExtensions case.

Yes, please follow-up with Blake in that bug.

Looks like bug 1277410 was where that work was being considered.

I'm dup-ing this to bug 1271553. There's a lot of good info in both bugs, but that one seems further along (has a patch).