Darin Fisher Reporter
	Description • 21 years ago

IPC extension to support distributed XPCOM

It should be possible (and perhaps straightforward) to use the IPC service's
message forwarding capability to implement a form of distributed XPCOM.

Some constraints/requirements:

 (1) Each process must have the same typelibs for all interfaces used.  Not a
     problem if only frozen interfaces are used.

 (2) Limit to synchronous method calls for simplicity.

 (3) Allow incoming method calls while waiting for a method call to return.
     This is necessary to avoid distributed deadlocks.

	Darin Fisher Reporter
	Comment 1 • 21 years ago

This tarball can be extracted into ipc/ipcd/extensions, and then the patch
named dcon-0.1.diff should be applied to ipc/ipcd to enable code.

	Darin Fisher Reporter
	Comment 4 • 21 years ago

This code is only marginally tested, and probably has bugs.  Also, I have not
yet implemented support for array parameters, and I need to setup an IPC client
observer so that I can prune out "remote instance wrappers" when a peer
abnormally terminates.

	Darin Fisher Reporter
	Comment 5 • 21 years ago

Please note that the v0.1 interface is a low-level interface.  It is totally
reasonable to imagine higher-level abstractions layered on top of this
interface.  The interface itself depends on the ClientID concept from
ipcIService.  The ipcIService can be used to query-by-name for the ClientID of a
connected client.

For instance, if a browser process informed the IPC daemon that it is named
"browser", then some other process could discover the ClientID of the browser
process at runtime, and use that in conjunction with ipcIDConnectService to
construct remote objects in that browser process.

Folks on IRC suggested a higher level API in which a "RemoteTarget" object could
be instantiated that would itself support the familiar nsIComponentManager and
nsIServiceManager.  This sounds like a good idea, and I'm keen on providing
something like this in addition to ipcIDConnectService.  At the low-level,
ipcIDConnectService is good because it ties in closely with the IPCD API.  This
dconnect extension is after all compiled into libipcdc.so!

The main question I have about this is how should we go about bootstrapping the
dconnect observer.  The observer needs to be started up, and hence the IPC
daemon itself has to be started up, in order to allow incoming requests from
other processes.  The v0.1 patch currently just forces things to startup at
xpcom-startup time.  This is fine for testing, but in the long run we might want
a better bootstrapping mechanism.  At the very least, we need an ENV var to
disable the bootstrapping process! ;-)

	Christian :Biesinger (don't email me, ping me on IRC)
	Comment 6 • 21 years ago

hmm, can you add a comment to ipcIDConnectService indicating whether it's
threadsafe?

	Darin Fisher Reporter
	Comment 7 • 21 years ago

Currently, it is not threadsafe... in part because the IPC API it not threadsafe.  
There are many challenges related to making this work correctly with multiple
threads, and i'm not sure that overcoming those challenges is worth the cost.

	Nicholas Allen
	Comment 8 • 21 years ago

Are threads affinitized either for making calls on objects or for serving requests?

	Darin Fisher Reporter
	Comment 9 • 21 years ago

> Are threads affinitized either for making calls on objects or for serving
requests?

All objects are bound to the main thread.  The remote objects are created in the
main thread of the remote process.  This is a simplification, and unless there
is great demand for MT support, I'd prefer to keep things simple.  However, I
should probably design the protocol to permit support for thread affinity in the
future.  An unused thread index field in the IPC messages (that must be 0 in
this version) might be sufficient to permit doing something more fancy in the
future.

	Darin Fisher Reporter
	Comment 10 • 20 years ago

the tarball portion of this patch was checked in under
ipc/ipcd/extensions/dconnect.  it is not built by default.

	Justin Wood (:Callek)
	Comment 11 • 20 years ago

Hmm...perhaps Item 1, could be crash-proof with an internal method to check the
*version* of called typelibraries, view a (small) dll of some sorts, with a
frozen tlb, that way even un-frozen tlb's can be checked against, (just a simple
number in the processes tlb gets modified, that the dll checks).  If it is the
same, it works, different it does not.

Possible?

	Darin Fisher Reporter
	Comment 12 • 20 years ago

> Hmm...perhaps Item 1, could be crash-proof with an internal method to check the
> *version* of called typelibraries, view a (small) dll of some sorts, with a

Actually, it just occured to me that the IID of the interfaces should be
sufficient to prevent version conflicts.  This assumes that everyone conforms to
the rule that IIDs must always be modified when interfaces change.

With the current patch, a mismatched IID should simply result in a
NS_ERROR_NO_INTERFACE exception.

	timeless
	Comment 13 • 20 years ago

checking iid's is a bad idea given our track record. if you want to support a 
sloppy mode which skips the more detailed typelib checking, that's fine, but 
i'd suggest that it not be the default.

	Darin Fisher Reporter
	Comment 14 • 20 years ago

(In reply to comment #13)
> checking iid's is a bad idea given our track record. if you want to support a 
> sloppy mode which skips the more detailed typelib checking, that's fine, but 
> i'd suggest that it not be the default.

Revising IIDs when interface change is an absolute MUST.  Mozilla
super-reviewers are aware of this requirement and are enforcing it.  And,
besides this only applies to non-frozen interfaces.  Frozen interfaces by
definition will be safe.

What further typelib checking is needed, really?

	timeless
	Comment 15 • 20 years ago

oh really? bug 33045 comment 44.

	Aleksey Nogin
	Comment 16 • 20 years ago

This have added 3 "might be used uninitialized" warnings on brad TBox:

+ipc/ipcd/extensions/dconnect/src/ipcDConnectService.cpp:1260
+ `nsresult rv' might be used uninitialized in this function
+
+ipc/ipcd/extensions/dconnect/src/ipcDConnectService.cpp:1546
+ `nsIInterfaceInfo*iinfo' might be used uninitialized in this function
+
+ipc/ipcd/extensions/dconnect/src/ipcDConnectService.cpp:1547
+ `PRUint8 paramCount' might be used uninitialized in this function

	Darin Fisher Reporter
	Comment 17 • 20 years ago

all three of those warnings are bogus in that they correspond to impossible
branches, but I'm not surprised that GCC is unable to determine that.  GCC 3.2.2
doesn't seem to complain.  anyways, i've checked in a fix.

also, i meant to comment in this bug that i have checked in Makefile changes
that enable this dconnect extension when the BUILD_DCONNECT environment variable
is set.  if BUILD_DCONNECT is not set, then the code is compiled but not linked
into the final shared library.  hence the reason why brad is compiling this code :-)

	Darin Fisher Reporter
	Comment 18 • 20 years ago

also, i fixed a crash that occurs because ipcMessageWriter does not properly
initialize itself when passed a non-zero initialCapacity in its constructor.

mvl: i think this is the crash you were seeing.

	Aleksander Adamowski
	Comment 19 • 19 years ago

Does this patch have a chance of being checked in? It blocks one of the more significant bugs in the Mozilla project.

	Darin Fisher Reporter
	Comment 20 • 19 years ago

Aleksander: I'm sorry, I just haven't had the time to work on this.  Moreover, since it isn't part of Firefox or Thunderbird, the need hasn't been compelling enough to take me away from work on Firefox or Thunderbird.  I can't really say when I will have a chance to work on this patch :-(

	Brendan Eich [:brendan]
	Comment 21 • 19 years ago

Not sure why this bug blocks bug 156493 -- Firefox wants out of process plugins I bet, as do many Gecko apps -- but distributed XPCOM isn't necessarily the way to get there from here.

/be

	Braden
	Comment 22 • 19 years ago

I think I'm responsible for that. If there's an easier, simpler, better path toward scriptable out-of-process plug-ins, I'm all for it. Given what I know about the existing machinery in the Mozilla code base, this seemed to me like the most reasonable solution. But someone better informed than I could very likely come up with a better idea.

I would, however, be disappointed to see this approach dismissed without another path to resolving bug 156493 clearly articulated.

	Ilya Konstantinov
	Comment 23 • 19 years ago

Brendan:
Plugins in Mozilla (unlike NS4) have access to XPCOM objects such as the DOM tree and the Service Manager.
"Distributed XPCOM" gives the ability to call methods in an XPCOM object living within a different process.

	Benjamin Smedberg
	Comment 24 • 19 years ago

Yes, I'm pretty sure this bug does block out-of-process plugins due to the plugin access to DOM interfaces.

	Darin Fisher Reporter
	Comment 25 • 19 years ago

Then that should make it pretty easy to negate the blocking1.8.1 request on bug 156493?  b/c there isn't much chance of the IPC module being ready for FF2.

	Mike Shaver (:shaver -- probably not reading bugmail closely)
	Comment 26 • 19 years ago

Which plugins make use of the DOM interface access?

	Darin Fisher Reporter
	Comment 27 • 19 years ago

> Which plugins make use of the DOM interface access?

I recall that flash does (in order to tweak the Accept header pref).  Doesn't the Java plugin use it in order to talk to the OJI layer in the browser?

	Mike Shaver (:shaver -- probably not reading bugmail closely)
	Comment 28 • 19 years ago

I thought the Java plugin was now a component, rather than something that uses the plugin API.  Didn't know about Flash, that's kinda too bad.

Wonder if we could just spot-fix the small number of APIs used by Flash and other such plugins to use something simpler than Fully General Distributed XPCOM, and then just remote the plugin API.

	Benjamin Smedberg
	Comment 29 • 19 years ago

The plugins actually provide new objects and APIs accessible to the DOM via XPCOM interfaces.

	Brendan Eich [:brendan]
	Comment 30 • 19 years ago

Cc'ing jst to set the record straight.

Our plugin API is built on the old NPAPI, with additions notably including NPRuntime, shipped in Firefox 1.0 and also supported in Safari (Opera too, I think).

We do not support the XPCOM plugins from 1999-2000 any longer.

That's my story and I'm sticking to it.

/be

	Benjamin Smedberg
	Comment 31 • 19 years ago

XPCOM plugins != NPAPI plugins that use XPCOM interfaces!

In particular, see http://lxr.mozilla.org/mozilla/source/modules/plugin/base/public/npapi.h#384
NPPVpluginScriptableInstance and NPPVpluginScriptableIID

Which allows NPAPI plugins to expose specialty interfaces to DOM scripts.

Also NPNVserviceManager and friends, which give NPAPI plugins access to the service manager and from there into anything-XPCOM.

	Johnny Stenback (:jst)
	Comment 32 • 19 years ago

AFAIK the current Java plugin is the only XPCOM plugin out there, but other plugins may indeed use XPCOM to do things in the browser. And any *scriptable* plugins available prior to Firefox 1.0 (i.e. flash, quicktime, and real IIRC) exposed an XPCOM interface for scripting.

Sun is hard at work at a new version of the Java plugin that is not an XPCOM plugin, and once that's available I think we should officially deprecate our support for XPCOM plugins, but I don't think we can easily take away the ability for plugins to get to the service manager and DOM nodes etc, that support has been around (and apparently even used by flash) for quite a few years now.