I think it would be worthwhile to investigate if JS plugins could run out of process by default. This may act as a testing ground for e10s, and improve performance for PDF.js and Shumway in Firefox. (In reply to John Schoenick [:johns] from Bug 558184 comment #29) > Since plugins will usually load in-content, making the frame load remote > would require nested-remote-iframes to be fully working first. Even then, > the plugin can easily use a stub as its handler that opens a remote iframe > and sends a content script to it to run OOP, so I think it'd be better to > let it be handled at that level. I don't know about the architecture involved here, but this sounds a bit too complicated to me (box in a box in a box). How are the non-JS plugins run oop currently? (Would this need some new [improved] NPAPI-like thing for the JS plugins?) Would it actually be worth it to find a common solution to this problem, or rather let any JS plugin find & implement an oop solution for itself? How likely is it that there will be more JS plugins (i.e. apart from PDF.js & Shumway), built by Mozilla or not (I can think of media decoders et al.)?

(CC :yury - Do you have any thoughts on the usefulness of doing this at a higher level vs letting shumway & co handle it?) To clarify this more: JS implemented plugins are fundamentally just iframes providing content, rather than boxes rendered to by an external module. The reason we need a API for this in bug 558184 is that plugin tags (applet/object/embed) need to have a special iframe-mode in order for them to load said content. <object> already supports things like <object data="http://example.com">, so has such a mode. Better yet, the implementation of all three tags is mostly shared, so it was a simple tweak to allow embed and applet to load iframe content. So the API we're adding is mostly just a way to register a handler for a MIME type that points to a URL, e.g. |application/x-shockwave-flash -> chrome://shumway/content/shumway-main.htm| So to run this OOP, the obvious way would be to load the object as a remote-iframe, which is how content processes are already implemented in gecko. But, this would mean that the js-plugin would need to be written to use fully async communication - or we'd need to proxy access with CPOWs. There was already a discussion about this in regards to shumway on dev-platform -- the desire for a worker that can be semi-synchronous to respond to sync APIs, which are needed to load flash. The reason I say this is better implemented at the plugin level is because its easy for js-plugins to do*: > <iframe mozbrowser src="chrome://shumway/content/shumway-content.xul"> And because the plugin would need to define how it wants to talk to its remote content, possibly with a shim in the non-remote portion to proxy communication. This also lets plugins control what they run OOP and what they run in process, rather than forcing the object to be a remote iframe, which would break things like implementing sync calls without further hacks (e.g. CPOWs) It's also not clear how much advantage OOP js-plugins give us - NPAPI plugins are OOP because they crash a lot, and are largely legacy implementations that run a lot of code on the main thread. Shumway and co. are free to use js-workers to multi-thread, and crashing is no more an issue than websites crashing, which would be mitigated by e10s. * Note that we can't use nested-remote-iframes yet, so that work would need to be finished first for this to be permissible in e10s mode

Yeah, that's pretty much what we are looking for. And OOP/e10s support will be a really nice addition. It will be nice to have non-remote iframe mode to toy with and debug sync calls at least for some time though. Let me briefly describe what PDF.js and Shumway are doing under-the-hood to expose couple of more items: 1) Using nsStreamConverter it hooks into HTTP request and feeds back an HTML page/viewer. The viewer in most of the cases has to have content permissions and be able to load additional resources from the resource URLs, e.g. initialize workers. 2) It needs to be aware in which window the HTML page was loaded to add listeners for sync/async communication. 3) It needs to be aware which OBJECT/EMBED tag has overlay to obtain PARAM tags or additional EMBED tag attributes, and also expose some JS APIs. 4) If native (alternative) plugin is available, it needs to be able to switch to the alternative plugin. We would expect the JS plugins to provide easier way to perform the steps above. Right now, PDF.js and Shumway contain the nsStreamConverter code that looks like a hack. The nsStreamConverter way does not really fit OBJECT/EMBED model (because of 2 and 3). Also, the request might be canceled (in favor of range requests) or not present if OBJECT/EMBED with empty src was used. Performing steps 1, 2, and 3 is error prone and might cause security issues and it will be even harder with OOP stuff. So it will be nice to have an API that encapsulates all of that.

(In reply to Yury Delendik (:yury) from comment #2) > Yeah, that's pretty much what we are looking for. And OOP/e10s support will > be a really nice addition. It will be nice to have non-remote iframe mode to > toy with and debug sync calls at least for some time though. > > Let me briefly describe what PDF.js and Shumway are doing under-the-hood to > expose couple of more items: > > 1) Using nsStreamConverter it hooks into HTTP request and feeds back an HTML > page/viewer. The viewer in most of the cases has to have content permissions > and be able to load additional resources from the resource URLs, e.g. > initialize workers. Could you explain this step more? the jsplugins patches will load the proper handler into the element as a frame, though currently you need to XHR or otherwise create a request for the src URI (window.frameElement.srcUI). Fixing that is possible with the jsplugins patches (though not done yet). OOP actually complicates this, because we'd need to proxy a HTTP stream to the content process that was opened in the parent. > 2) It needs to be aware in which window the HTML page was loaded to add > listeners for sync/async communication. Is there anything complicated here besides looking at window.frameElement? > 3) It needs to be aware which OBJECT/EMBED tag has overlay to obtain PARAM > tags or additional EMBED tag attributes, and also expose some JS APIs. Though it didn't have jsplugins in mind, bug 853995 will expose an API on nsIObjectLoadingContent to query parsed <params> so the plugin doesn't need to emulate it (e.g. window.frameElement.getPluginParameters) > 4) If native (alternative) plugin is available, it needs to be able to > switch to the alternative plugin. The jsplugins patch will provide window.frameElement calls to both check and trigger native fallbacks > We would expect the JS plugins to provide easier way to perform the steps > above. Right now, PDF.js and Shumway contain the nsStreamConverter code that > looks like a hack. The nsStreamConverter way does not really fit > OBJECT/EMBED model (because of 2 and 3). Also, the request might be canceled > (in favor of range requests) or not present if OBJECT/EMBED with empty src > was used. > > Performing steps 1, 2, and 3 is error prone and might cause security issues > and it will be even harder with OOP stuff. So it will be nice to have an API > that encapsulates all of that. With the exception of 1, which will need a follow-up bug to jsplugins to allow passing a stream to the handler, I don't see what OOP gets us here, other than the plugin being able to mix sync APIs and workers on one thread via CPOWs. @yury Do you have an idea of what this sync API would look like? It still seems to me, that for the OOP portion, shumway just creating an <iframe mozbrowser> would let it handle sync calls however it chooses in the parent, and proxy messages to its OOP frame as necessary -- I'm not sure how the browser would make a one-size-fits-all API for that.

This bug is not a priority and I encourage people not to spend time on it, unless somehow this is a better way to get shumway working on device than the current plan which is to use workers.

> > 1) Using nsStreamConverter it hooks into HTTP request and feeds back an HTML > > page/viewer. The viewer in most of the cases has to have content permissions > > and be able to load additional resources from the resource URLs, e.g. > > initialize workers. > > Could you explain this step more? the jsplugins patches will load the proper > handler into the element as a frame, though currently you need to XHR or > otherwise create a request for the src URI (window.frameElement.srcUI). > Fixing that is possible with the jsplugins patches (though not done yet). > OOP actually complicates this, because we'd need to proxy a HTTP stream to > the content process that was opened in the parent. I see. It's probably will be applicable only for fullscreen plugins. To detect the content type, the initial HTTP request is created. And it will be useful to keep this request opened and handle it e.g. at the content proxy/process opened in the parent (?) and just pass it to the OOP window. We can send the XHR request(s), but ideally people will expect only one request. > > > 2) It needs to be aware in which window the HTML page was loaded to add > > listeners for sync/async communication. > > Is there anything complicated here besides looking at window.frameElement? Probably not a problem here. I just wanted to demonstrate that some chrome privileged content might communicate with the parent and/or viewer, e.g. the viewer might communicate with Firefox's find toolbar. > > @yury Do you have an idea of what this sync API would look like? It still > seems to me, that for the OOP portion, shumway just creating an <iframe > mozbrowser> would let it handle sync calls however it chooses in the parent, > and proxy messages to its OOP frame as necessary -- I'm not sure how the > browser would make a one-size-fits-all API for that. On parent side the following can be executed: `var result = obj1.callback1('arg', 1);`. On viewer side: `var result = execInWindow("window.navigator.userAgent");`. I realize that this communication will be performed between parent's window and viewer's worker and this is a scope of the bug 930908, but that's something to keep in mind (e.g. if it's possible to enable the same thing for the OOP jsplugin stuff).

(In reply to Yury Delendik (:yury) from comment #5) > I see. It's probably will be applicable only for fullscreen plugins. To > detect the content type, the initial HTTP request is created. And it will be > useful to keep this request opened and handle it e.g. at the content > proxy/process opened in the parent (?) and just pass it to the OOP window. > We can send the XHR request(s), but ideally people will expect only one > request. We already duplicate requests for click-to-play, but this can be solved with the new API. (the new API already otherwise handles full-page plugins properly) > Probably not a problem here. I just wanted to demonstrate that some chrome > privileged content might communicate with the parent and/or viewer, e.g. the > viewer might communicate with Firefox's find toolbar. Ideally, shumway wouldn't run all of its code as privileged just because it is loaded in extension mode -- the jsplugins api would let you load a non-privileged frame and just expose a shumway-extension API to it with a limited set of privileged functions. > On parent side the following can be executed: `var result = > obj1.callback1('arg', 1);`. On viewer side: `var result = > execInWindow("window.navigator.userAgent");`. I realize that this > communication will be performed between parent's window and viewer's worker > and this is a scope of the bug 930908, but that's something to keep in mind > (e.g. if it's possible to enable the same thing for the OOP jsplugin stuff). So unless it turns out to be easier to implement bug 930908 by just hosting shumway in a content process with re-entrant CPOWs, I don't see how OOP gives us any advantages here. Even then, adding it to the platform API only saves shumway a few lines of code. Do you see any compelling reasons to build in jsplugin-oop at the platform level besides as a strategy for implementing bug 930908? The streamlistener stuff needs to be solved, but is actually harder with OOP rather than easier.

We need to solve the stream issue in bug 801406 anyway so that CtP works properly with POSTdata.

> So unless it turns out to be easier to implement bug 930908 by just hosting shumway in a content process with re-entrant CPOWs, I don't see how OOP gives us any advantages here So, fast forward six months, and we now have re-entrant two way CPOWs! It came up in the e10s meeting that using these was in fact being considered instead of ShumwayWorker. If we think having this happen at the API level is useful this bug might still make sense. It looks like we'll be using an actual iframe in the core jsplugins bug, so supporting this at the API level would essentially just be a bool to request that we stick the remote flag on your iframe.

That's pretty amazing timing - just yesterday I filed bug 1030721 about this. (In reply to John Schoenick [:johns] from comment #8) > It looks like we'll be using an > actual iframe in the core jsplugins bug, so supporting this at the API level > would essentially just be a bool to request that we stick the remote flag on > your iframe. Would this enable us to use the same process for all iframes? That seems to be a hard requirement because otherwise even the simplest banners use cases will kill us on the memory usage and startup time front.

Multiple content processes are not supported in e10s yet, so there's no scheduling of what remote frame goes to what process, but when that's implemented, I imagine it would be same-origin based (or at least opt-in to that behavior)

Hmm, how would this interact with e10s windows right now, then? Would it run in the same process after all? Other than that, I guess it would be ok to not have explicit control over which process Shumway script execution ends up in, at least as a first step. There are some features around inter-instance communication and local storage that are likely to require having all SWFs run in the same process, but we can punt on getting those correct for now. And in the final setup, could we also have the main iframe be in the same process as the surrounding page, with a nested iframe being remote? That'd enable us to have the rendering in the page content's process and script execution in the remote one, with all the parallelization advantages that has.

(In reply to Till Schneidereit [:till] (on vacation July 19th to August 14th) from comment #11) > Hmm, how would this interact with e10s windows right now, then? Would it run > in the same process after all? I'm not sure that current e10s has working support for remote-iframes inside already-remote pages, but once multiple-content-processes work my impression was that this was reasonably possible. > And in the final setup, could we also have the main iframe be in the same > process as the surrounding page, with a nested iframe being remote? That'd > enable us to have the rendering in the page content's process and script > execution in the remote one, with all the parallelization advantages that > has. The jsplugins api in bug 558184 will work by nesting a anonymous iframe inside whatever <embed>/<object> tries to load your plugin, and loading your plugin's handler into it. Your handler could further nest a <iframe mozbrowser> to then create its own remote iframe and manage everything from there, with nothing further needed. This bug was opened about baking this OOP behavior into the API somehow, and I mentioned in comment 1 that it might just be better handled by the individual plugin doing it. So for actually implementing this in shumway, I don't think we strictly need anything from this bug, or even the jsplugins API -- shumway would just need to create itself a remote iframe and go. If shumway wants to try this, we should open a DOM bug around improving remote-iframe support to fit those needs, and then use this bug to consider what the jsplugins API could provide to make this easier, if anything.

bz, do you think this bug is useful on its own, or should it be duped to bug 558184?

Assigning to bz for now because the out-of-process work for jsplugins is part of bug 558184.

jsplugins will also be OOP.