As part of the WebExtension support in Bug 1518841, we need a way to let embedders send and receive messages from content through WebExtensions.

Hi :agi, I'm interested in following this work for the reasons in Bug 1500828. Are there existing API docs that could give me a bit of an overview of what the final experience will look like here?

+:stomlinson and :vladikoff as it looks like this will be the path we need for integrating with webchannel messages from FxA in Fenix.

Hi :cpeterson and :agi, is the plan to use the same WebChannel abstraction [1] that is used in desktop? I'm not sure what your requirements are, WebChannels are a way for trusted embedders to send & receive messages from the browser.

[1] - https://developer.mozilla.org/en-US/docs/Mozilla/JavaScript_code_modules/WebChannel.jsm

(In reply to Shane Tomlinson [:stomlinson] from comment #3)

Hi :cpeterson and :agi, is the plan to use the same WebChannel abstraction [1] that is used in desktop?

I don't know. That's a questions for Agi. :)

Adding [geckoview:fenix:m2] whiteboard tag because the Fenix team says they would like to start using WebExtensions for ad blocking and reader mode in their Fenix M2 or M3 milestones.

Sorry for the delay, I'm still trying to figure out what to do here.

I've looked around a little bit and :snorp idea of using a native channel in Bug 1500828 sounds good to me.

• We could expand Native Messaging on android letting apps define a set of native apps manifests in a folder inside the apk, e.g. in /assets/native_apps/.

  • In the future, if we want to allow users to install custom native apps we could do so allowing a /sdcard location for manifests too.

• For our use case, a geckoview app would define a "browser" native app which is used for internal web extensions to talk to geckoview and the app.

  • The path member of the native manifest would be unused for now.
  • All messages would go through the WebExtension object with a native messaging delegate, the app would be able to forward it to other apps if needed (how is TBD).

public class WebExtension {
    public static class NativeApp {
        String name;
        String description;
        // String path; unused for now
        List<String> allowedExtensions;
    }
    public static class NativeMessagingDelegate {
        void onNativeAppRegistered(NativeApp);
        // Redirects to other apps if needed
        GeckoResult<String> onMessage(String app, String json); // Maybe JSON object?
    }
    void setNativeMesssagingDelegate(NativeMessagingDelegate);
    GeckoResult<String> sendNativeMessage(String app, String json);
}

• On the extension side, the "browser" native app would look like an ordinary one:

var port = browser.runtime.connectNative("browser");

/*
Listen for messages from the app.
*/
port.onMessage.addListener((response) => {
  console.log("Received: " + response);
});

// Send message to GeckoView
port.postMessage("ping from web extension");

This looks pretty close to what I was imagining. I would use JSONObject instead of String for values. I think for simplicity sake, though, maybe we shouldn't actually use the native messaging API and instead add some other way for extensions to get the Port. Maybe a special extension id, like browser.runtime.connect({ name: 'app' })? Kris, I'm sure you have an opinion here?

I like that onMessage and sendNativeMessage return GeckoResult, but AFAIK the extension messaging is one way, so these probably don't make sense. We could make sendNativeMessage return GeckoResult<Void> to indicate that the message was delivered, but we don't really do that anywhere else (yet).

(just repeating what I said on slack)

I have a mild preference for native messaging because the app is more like a native app than an extension. We could provide a hard-coded native app to avoid having people add a manifest.

I like that onMessage and sendNativeMessage return GeckoResult, but AFAIK the extension messaging is one way, so these probably don't make sense. We could make sendNativeMessage return GeckoResult<Void> to indicate that the message was delivered, but we don't really do that anywhere else (yet).

Ah you're right. I was fooled from runtime.sendMessage from which you can return a Promise as a response: https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/runtime/onMessage#Sending_an_asynchronous_response_using_a_Promise

I would use JSONObject instead of String for values

FWIW, with WebChannels for FxA we started out sending JSON objects from content to chrome code, but were advised to only send strings for security reasons: https://bugzilla.mozilla.org/show_bug.cgi?id=1238128

The story may be different here of course, just sharing for related historical context.

On a conversation with :snorp and :sebstian on Slack, it came up that we probably want messaging from within content scripts too directly, and the possibility to map the message back to a GeckoSession instance.

Using runtime.sendMessage with a custom extensionId="browser" or similar, would be pretty nice as:

• It's vailable from content scripts
• The meassage should be easily mapped back to a GeckoSession
• (minor but) It has a return value so it could embed a GeckoResult response (see comment #7 and comment #8).

The drawback is that we would need to have a "magic" value for extensionId.

I'm going to look at the code a little bit to see if this approach is feasible.

The approach in Comment #10 seems to work well. Looks like all we need is a MessageChannel that responds to browser for sending messages. I'm going to look next into receiving messages in web extensions and hopefully establishing channels too.

[geckoview:fenix:m2] not [geckoview:fenix:p1]

One thing I was thinking about, we probably want to add a new permission to web extensions to explicitly expose this feature.

Which makes me think we should not use sendMessage and just have a new API tied to the pref that uses it, we already pretty much don't do anything in sendMessage('browser' other than just redirecting the message to the java layer.

It would be something like runtime.sendBrowserMessage(...) tied to the geckoviewAddons permission.

This would work well for 3rd party extension too, because the GeckoView app would just check that external extensions don't have this permission in the manifest.

:kmag could you look at the above comment to see if it makes sense to you?

I'm proposing to add a new Extension API to runtime called sendBrowserMessage(...) similar to runtime.sendMessage that would be used by GeckoView apps to exchange message between the app layer and content scripts. This API would be behind a privileged pref geckoviewAddons.

We would need a pref because a GeckoView app might have both internal extensions that use browser messages and external extension which shouldn't have access to browser messages.

(In reply to Agi | :agi | ⏰ EST | he/him from comment #14)

We would need a pref because a GeckoView app might have both internal extensions that use browser messages and external extension which shouldn't have access to browser messages.

Would this API only be available to extensions? Would we be able to build a WebChannel abstraction [1] on top of this scheme? I ask because FxA needs a way to communicate with Fenix as part of the signin/signup for Sync process.

[1] - https://developer.mozilla.org/docs/Mozilla/JavaScript_code_modules/WebChannel.jsm

(In reply to Shane Tomlinson [:stomlinson] from comment #15)

(In reply to Agi | :agi | ⏰ EST | he/him from comment #14)

We would need a pref because a GeckoView app might have both internal extensions that use browser messages and external extension which shouldn't have access to browser messages.

Would this API only be available to extensions? Would we be able to build a WebChannel abstraction [1] on top of this scheme? I ask because FxA needs a way to communicate with Fenix as part of the signin/signup for Sync process.

[1] - https://developer.mozilla.org/docs/Mozilla/JavaScript_code_modules/WebChannel.jsm

WebExtensions is the way we envision GV apps like Fenix will be interacting with content, so yes it will be only available to Extensions, but we expect Fenix to implement something similar to WebChannel with a WebExtension.

From a cursory look it should be very straightforward to implement WebChannel as a WebExtension with the help of messaging that is implemented in this bug.

I'll make sure I have a prototype for WebChannel as a WebExtension before landing this.

A-C will use this extension messaging API to implement Reader View:

https://github.com/mozilla-mobile/android-components/issues/1346
https://github.com/mozilla-mobile/android-components/issues/1385

Little API update on this.

WebExtension API

There are two new runtime WebExtensions API being added as part of this, both of these APIs will be behind geckoviewAddons pref which is only available to builtIn WebExtensions (i.e. no third-party web extensions).

• runtime.sendBrowserMessage: similar to runtime.sendMessage, sends a message to the app and optionally gets a response Promise that is sent back to the WebExtension
• runtime.connectBrowser: similar to runtime.connect, will connect to the app and return a Port object that allows for bidirectional communication.

e.g. to send a simple message a web extension can do:

let response = browser.runtime.sendBrowserMessage('Ping from WebExtension');
response.then(message => {
    console.log(`Message from app: ${message}`);
});

to establish bidirectional communication:

let port = browser.runtime.connectBrowser();
port.onMessage.addListener(message => {
  console.log(`Message from port ${message}`);
});
port.postMessage("Ping from content");

GeckoView Java API

WebExtension has a new property called messageDelegate that allows GeckoView apps to respond to and send messages

class WebExtension {
    String location;
    String id;
    MessageDelegate messageDelegate; // new
}

interface MessageDelegate {
    GeckoResult<Object> onMessage(WebExtension source, Object message, GeckoSession session);
    void onConnect(WebExtension source, Port port, GeckoSession session);
}

the Object message can either be a primitive type, e.g. for sendBrowserMessage('test') the message would be a String, or a JSONObject if the message is a complex javascript object.

onConnect returns an instance of Port (described below) that can be used to send and receive messages from the WebExtension.

Port is the java equivalent of the WebExtension Port:

class Port {
    void postMessage(JSONObject message);
    void setDelegate(PortDelegate delegate);
}

interface PortDelegate {
    void onPortMessage(WebExtension source, Object message, GeckoSession session);
    void onDisconnect(WebExtension source, Port port);
}

onPortMessage is similar to onMessage except it cannot send responses.

E.g. for a simple single-tab app:

public class MyActivity extends AppCompatActivity implements WebExtension.MessageDelegate, WebExtension.PortDelegate {
    private WebExtension.Port mContentPort;

    @Override
    public void onConnect(@NonNull WebExtension source,
                          @NonNull WebExtension.Port port,
                          @Nullable GeckoSession session) {
        try {
            mContentPort = port;
            mContentPort.setDelegate(this);
            mContentPort.postMessage(new JSONObject("{\"message\": \"ping from java\"}"));
        } catch (JSONException ex) {
            throw new RuntimeException(ex);
        }
    }

    @Override
    public void onDisconnect(@NonNull WebExtension source, @NonNull WebExtension.Port port) {
        if (mContentPort == port) {
            mContentPort = null;
        }
    }

    @Override
    public void onPortMessage(@NonNull WebExtension source, @NonNull Object message,
                              @Nullable GeckoSession session) {
        Toast.makeText(getApplicationContext(), message.toString() + " " + session,
                Toast.LENGTH_SHORT).show();
    }
}

Where mContentPort can be used at any time to send messages to the WebExtension for a specific GeckoSession instance.

I'm considering to build something with GeckoView and this API raises a question for me. From how I understand the current GeckoView API, The GeckoSession represents one currently loaded site in Gecko, like one Browser tab in Firefox Desktop. GeckoRuntime on the other hand represents the entire Gecko instance for my app. If this new WebExtensions API associates messages and message channels with one GeckoSession like the method signatures show, how would I communicate with my web extension background script / page? Would there be a new session just for the background script?

(In reply to Jovan Gerodetti from comment #19)

how would I communicate with my web extension background script / page? Would there be a new session just for the background script?

Messages from background scripts are handled the same way, simply the session parameter would be null in that case.

Try looks pretty green: https://treeherder.mozilla.org/#/jobs?repo=try&revision=f40c3e7bd3818d154f9e895db919c177ecfc00b9

Depends on D22620

This change adds runtime.sendBrowserMessage and runtime.connectBrowser to
the WebExtension API.

This APIs are available behind the new privileged-only permission
geckoviewAddons and are used by GeckoView apps to communicate between content
and the app.

Depends on D22621

Depends on D22620

67=wontfix. Fenix MVP will use GeckoView 68, so we don't need to uplift this fix to 67 Beta.

https://hg.mozilla.org/mozilla-central/rev/48f975c56cc6
https://hg.mozilla.org/mozilla-central/rev/4a6e7c0051cb

only half of the patches landed, sorry for the noise

https://hg.mozilla.org/mozilla-central/rev/8fd075dbd205
https://hg.mozilla.org/mozilla-central/rev/5e0478607244

Is there already a good example of how we can use this? https://mozilla.github.io/geckoview/consumer/docs/web-extensions is out-dated in my opinion.

I tried to convert the above example to use with the new API, but I did not succeed.

I have to communicate from my own web application to the Android code and back.

What I already did is create a background.js script (where I created a port by calling browser.runtime.connectNative) and a content.js script (which can access the DOM of my application) which can receive messages by browser.runtime.onMessage.addListener.
But the part that is missing is how I can call/access the 'port.postMessage' from my content.js script (because the port is created in the background.js script).

Thanks in advance!

Moving some WebExtension bugs to the GeckoView::Extensions component.