Closed Bug 1396172 (webext-tb) Opened 7 years ago Closed 5 years ago

[meta] Add support for WebExtensions

Categories

(Thunderbird :: Add-Ons: Extensions API, enhancement)

Product:
Thunderbird
Component:
Add-Ons: Extensions API
Type:
enhancement
Priority:
Not set
Severity:
normal

Tracking

(thunderbird59+ affected, thunderbird60 affected)

Status:
RESOLVED FIXED
Tracking Flags:
Tracking Status
thunderbird59 + affected
thunderbird60 --- affected

People

(Reporter: public, Unassigned)

References

(Depends on 11 open bugs, Blocks 1 open bug)

Details

(Keywords: meta)

Attachments

(1 file, 3 obsolete files)

Part 1: Content policy [landed comment #13]
(deleted), patch
jorgk-bmo
: review+
Details | Diff | Splinter Review
User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Firefox/52.0 Build ID: 20170802111520 Steps to reproduce: Tried to install a few WE: https://addons.mozilla.org/en-US/firefox/addon/ublock-origin/ https://addons.mozilla.org/en-US/firefox/addon/scroll_anywhere/ Actual results: I could not install them. Expected results: Some Firefox are really useful for Thunderbird, but Firefox 57 will only run WebExtensions. Can you please add support for WebExtensions in Thunderbird?
Only then can uBlock Origin be supported for Thunderbird: https://github.com/gorhill/uBlock/issues/117
Component: Untriaged → General
See bug 1346918 for tests we have switched off due to Webextensions not being supported in TB. I think that's a little out of date, so the switched-off tests can be found more easily here: http://searchfox.org/mozilla-central/search?q=appname+%3D%3D+%22thunderbird%22&case=false&regexp=false&path= http://searchfox.org/mozilla-central/search?q=MOZ_APP_NAME+%3D%3D+%22thunderbird%22&case=false&regexp=false&path=
See Also: → https://bugzilla.mozilla.org/show_bug.cgi?id=1346918
I was sure I had already filed a bug for this, but it seems not. Anyway, confirming we want to at least enable the basics so that we can benefit from some of the features it brings. WebExtensions most likely will not work out of the box for Thunderbird, and to properly support some of the UI facing features we're going to have to invest quite some work (and copy/paste a lot of code, sadly). I'm going to start this out with a WiP patch that makes WX not break on the content policy we have set. This might be enough to re-enable the tests. As to what is required to make the UI features work, we'll need to create implementations of some of the ext- scripts in browser/ https://dxr.mozilla.org/comm-central/search?q=path%3Abrowser%2Fcomponents%2Fextensions+path%3Aext-&redirect=false This shouldn't be too hard for browser action buttons. The tabs/windows implementation will be a lot of work, and it is unclear how we want to map tabs (either content tabs in our tabmail implementation, or message readers, or both. Probably not xul tabs, except maybe to the extent that special tabs can be created and closed. Content scripts should not be able to run there). There are also some APIs we probably don't want to implement or can't (devtools, omnibox, settings/url overrides). Then of course we can start creating new APIs to suit mail features, but that should be coordinated and APIs should not be accepted lightly.
Status: UNCONFIRMED → NEW
Ever confirmed: true
Attached patch WiP patch for basic features - v1 (obsolete) (deleted) — — Splinter Review
Jörg, when you are back at work and have some spare time, maybe you can do a try run with this patch and those wx tests enabled again. I suspect at least bug 1267612 might work again this way.
(BMO only inserts a link if the bug and the number are on the same line: bug 1267612). Spare time? Never. That said, I can easily enable the disabled tests (which can be found with the searches from comment #2) and see what still fails.
OK, running a few disable tests by hand: mozilla/mach xpcshell-test toolkit/mozapps/extensions/test/xpcshell/test_webextension_install.js fails with TEST_STATUS: Thread-1 test_implicit_id FAIL [test_implicit_id : 25] Add-on signing is enabled - false == true Hmm, not strictly a w/e failure. We could submit a patch to enable the test but skip that check. mozilla/mach xpcshell-test toolkit/components/extensions/test/xpcshell/test_csp_custom_policies.js Passes. Nice. mozilla/mach xpcshell-test toolkit/mozapps/extensions/test/xpcshell/test_webextension_icons.js Passes. toolkit/mozapps/extensions/test/xpcshell/test_webextension.js still fails. toolkit/mozapps/extensions/test/xpcshell/test_webextension_embedded.js still fails. mozilla/mach xpcshell-test toolkit/mozapps/extensions/test/xpcshell/test_webextension_install_syntax_error.js Passes. toolkit/mozapps/extensions/test/xpcshell/test_ext_management.js hangs. I'll get this landed and create a patch to re-enable some tests.
Attached patch 1396172-re-enable-tests.patch (obsolete) (deleted) — — Splinter Review
These are the tests we can enable. toolkit/components/extensions/test/xpcshell/test_csp_custom_policies.js is different since there the entire directory is disabled: http://searchfox.org/mozilla-central/rev/01970ed92d74f82d4e94a1e4365892bbcc593889/toolkit/components/extensions/test/xpcshell/xpcshell.ini#4 So in order to re-enable it, we'd have to re-enable the entire directory and then disable everything that fails. Here's a patch for the easy stuff in toolkit/mozapps/extensions/test/xpcshell.
Attachment #8918600 - Flags: feedback?(philipp)
Attachment #8918600 - Flags: feedback?(kmaglione+bmo)
Attachment #8918600 - Flags: feedback?(kmaglione+bmo) → feedback+
Comment on attachment 8918600 [details] [diff] [review] 1396172-re-enable-tests.patch Thanks Kris. I need r+ to get that landed ;-) Any idea why I need to disable some sub-tests?
Attachment #8918600 - Flags: review?(kmaglione+bmo)
Here we have a try with both patches and also re-enabling tests in toolkit/components/extensions/test/xpcshell: http://searchfox.org/mozilla-central/rev/01970ed92d74f82d4e94a1e4365892bbcc593889/toolkit/components/extensions/test/xpcshell/xpcshell.ini#4 test_csp_custom_policies.js should pass but there will be many tests that don't pass there. Try: https://treeherder.mozilla.org/#/jobs?repo=try-comm-central&revision=1b752dbabc80a2955e3a970a138d2a949eef95e2
Attachment #8918594 - Attachment is obsolete: true
Attachment #8918621 - Flags: review+
Comment on attachment 8918600 [details] [diff] [review] 1396172-re-enable-tests.patch I think the pattern that is more common is: add_task({ skip_if: () => mozinfo.appname == "thunderbird" }, async function test_the_thing() { ... }); This way the test doesn't even show in the list and you don't need to load appconstants. Untested though.
Attachment #8918600 - Flags: feedback?(philipp) → feedback+
Pushed by mozilla@jorgk.com: https://hg.mozilla.org/comm-central/rev/c91a3bb11073 Add support for WebExtensions: Part 1: Content Policy. r=jorgk
Keywords: checkin-needed
(In reply to Jorg K (GMT+2) from comment #9) > Try: > https://treeherder.mozilla.org/#/jobs?repo=try-comm- > central&revision=1b752dbabc80a2955e3a970a138d2a949eef95e2 That didn't work at all due to some infra problem, so here again: https://treeherder.mozilla.org/#/jobs?repo=try-comm-central&revision=f97a372cb31aca6f4ace33b3031532f9f5799a7f
Attachment #8918621 - Attachment description: Part 1: Content policy → Part 1: Content policy [landed comment #13]
Re-enabling tests in toolkit/components/extensions/test/xpcshell doesn't appear to be a good idea since 55 of them time out and another 5 fail: https://archive.mozilla.org/pub/thunderbird/try-builds/mozilla@jorgk.com-f97a372cb31aca6f4ace33b3031532f9f5799a7f/try-comm-central-linux64/try-comm-central_ubuntu64_vm_test-xpcshell-bm54-tests1-linux64-build9.txt.gz So we need to invest more work so see why those fail. For now, we can only enable three tests as per attachment 1396172-re-enable-tests.patch. Kris, perhaps you can give us some details of what we need to do to get TB "WebExtension" ready. As a start, we don't want to define our own interfaces, but only run "normal" FF W/E like uBlock.
Attachment #8918600 - Flags: review?(kmaglione+bmo) → review+
(In reply to Jorg K (GMT+2) from comment #8) > Comment on attachment 8918600 [details] [diff] [review] > 1396172-re-enable-tests.patch > > Thanks Kris. I need r+ to get that landed ;-) > Any idea why I need to disable some sub-tests? Sorry for the delay. I was sick most of last week; didn't get much done. I'm not sure why the permissions tests don't work. How were they failing? The other tests depend on the new signing system, which as far as I know Thunderbird still doesn't implement.
Attached file 1396172-failures.txt (obsolete) (deleted) —
Kris, thanks for the comments. I'm not sure what you're referring to with > The other tests depend on the new signing system ... You're referring to test_implicit_id[_temp] or the tests in toolkit/components/extensions/test/xpcshell which mostly time out or fail? My patch disables four tests: test_implicit_id, test_implicit_id_temp, test_permissions_prompt and test_permissions_prompt_cancel. Assuming that you were referring to the former two, I enabled the two permission tests again to check how they fail. mozilla/mach xpcshell-test toolkit/mozapps/extensions/test/xpcshell/test_webextension_install.js fails with: 0:05.90 TEST_STATUS: Thread-1 test_permissions_prompt FAIL [test_permissions_prompt : 522] Permission handler was invoked - "undefined" != "undefined" Full log enclosed. Do you think those two can be made to pass?
Flags: needinfo?(kmaglione+bmo)
Requesting tracking as without having basic support for WebExtensions, add-ons will now not be able to have preference displays, as the legacy preference bindings have been removed from toolkit (bug 1414406).
tracking-thunderbird59: --- → ?
Blocks: 1418914
(In reply to Mark Banner (:standard8) from comment #18) > Requesting tracking as without having basic support for WebExtensions, > add-ons will now not be able to have preference displays, as the legacy > preference bindings have been removed from toolkit (bug 1414406). Ah I thought this was just about the inline options feature, not the very legacy way of add-ons showing preferences. To be able to provide compat for existing add-ons using inline options we might need to resurrect those bindings in c-c and hope that the underlying xbl stuff is not removed before 59. With the patch landed in comment 13, it is possible to install WebExtensions in Thunderbird actually. What is not working is any kind of API that is in browser/ which also includes the tabs API, and we don't have any mail-specific APIs. I don't see that we will have the energy to provide this, and transition our add-ons in time for 59, so I fear we'll have to do everything possible to keep legacy add-ons running. Jörg, as you approved tracking for 59, what is the scope you expect for this bug?
(In reply to Philipp Kewisch [:Fallen] from comment #19) > Jörg, as you approved tracking for 59, what is the scope you expect for this > bug? Hmm, I'm perhaps just ignorant about the difference of "tracking ?" and "tracking +". I guess it's an item we need to keep an eye on for TB 59. I don't have a scope in mind. Previously tracking flags were mostly cosmetic, we have shipped ESR versions without tracked bugs being fixed. This time we won't have that option since if we're not fixing the add-on issues I've just summaries on Maildev "Add-ons under threat in TB", TB 59 will just be unusable. Also note bug 1418914.
(In reply to Philipp Kewisch [:Fallen] from comment #19) > (In reply to Mark Banner (:standard8) from comment #18) > > Requesting tracking as without having basic support for WebExtensions, > > add-ons will now not be able to have preference displays, as the legacy > > preference bindings have been removed from toolkit (bug 1414406). > > Ah I thought this was just about the inline options feature, not the very > legacy way of add-ons showing preferences. To be able to provide compat for > existing add-ons using inline options we might need to resurrect those > bindings in c-c and hope that the underlying xbl stuff is not removed before > 59. When I said that, I hadn't actually realised this bug had already landed, and it was just the tests to fix up. It turns out that it is actually bug 1418914 that is preventing from a WebExtension being used for in-line preferences. Resurrecting the xbl stuff might be an option, but I suspect isn't sustainable, and may be hard to hook into the add-ons manager (especially now the support for the install.rdf options is also removed, so you can't install an add-on declaring the old-style inline preferences). > With the patch landed in comment 13, it is possible to install WebExtensions > in Thunderbird actually. What is not working is any kind of API that is in > browser/ which also includes the tabs API, and we don't have any > mail-specific APIs. We don't need mail specific APIs. We can use hybrid extensions (aka embedded WebExtensions), https://developer.mozilla.org/Add-ons/WebExtensions/Embedded_WebExtensions. It is a little more complicated in some areas, but it is possible to do, and a good transition stage if the long-term furture might be for WebExtensions.
Alias: webext-tb
Flags: needinfo?(kmaglione+bmo)
Keywords: leave-open → meta
Magnus, I'd suggest we create a WebExtensions component in Thunderbird. Thoughts?
Flags: needinfo?(mkmelin+mozilla)
Sure, could be useful.
Flags: needinfo?(mkmelin+mozilla)
Blocks: webext
Depends on: 1468667
Depends on: 1469238
(In reply to Mark Banner (:standard8) (away 13 - 17 Aug) from comment #21) > We don't need mail specific APIs. Of course not directly. So far even the "Browser API" isn't completed. But is there any chance that there will be "mail specific" APIs in the future? I think there should be at least an *optional* possibility to have some API for the most common Addon types. IMHO the current "API", which (like in pre-WebExtension Firefox versions) are just the internal interfaces which tend to change regularly, is pretty horrible. Especially for beginners or everyone without deep insight in the backend code.
(In reply to Manuel Reimer from comment #24) > (In reply to Mark Banner (:standard8) (away 13 - 17 Aug) from comment #21) > > We don't need mail specific APIs. > > Of course not directly. So far even the "Browser API" isn't completed. My comment was taken slightly out of context - the comment was with respect to being able to do various things easily in Thunderbird 60, so that extensions could easily upgrade and get working again (e.g. preferences). However, other workarounds were made and there was enough for extensions without needing WebExtension parts (and I could never get it to work reliably with a hybrid version anyway...). Off-topic, but the browser API is never going to be "completed" - there will always be more features to add, or different extension points that people want. > But is there any chance that there will be "mail specific" APIs in the > future? I think there should be at least an *optional* possibility to have > some API for the most common Addon types. Well if you look at the dependencies of this bug, you'll see there's already a couple mentioned... IMO it is unclear what is left in this bug to do - I believe the basic WebExtension support is there now, maybe this should be closed, and a new bug set up to track the implementation of various web extension APIs.
This is actually the meta-but for WebExtensions. Given the amount of comments and patches landed we could consider creating a new one though.
Summary: Add support for WebExtensions → [meta] Add support for WebExtensions
No longer blocks: 1418914
Depends on: 1418914
Attachment #8922244 - Attachment is obsolete: true
Comment on attachment 8918600 [details] [diff] [review] 1396172-re-enable-tests.patch Never landed, a more comprehensive patch landed in bug 1468667.
Attachment #8918600 - Attachment is obsolete: true
Depends on: 1487008
Depends on: 1487017
Component: General → Add-Ons: Extensions API
Depends on: 1499617
Here's a rough list I wrote the other day of the next stuff I'm working on. Everything under mailTab now has some more meat on the bones in bug 1499617; the accounts and messages I'll be working on in bug 1488176. https://gist.github.com/darktrojan/d5102d4505298edc4535528ea211f6c6
Depends on: 1503421
Depends on: 1503423
Depends on: 1504475
Depends on: 1520338
Depends on: 1520367
Depends on: 1531591
Depends on: 1531593
Depends on: 1531317
Depends on: 1531597
Depends on: 1538590
Depends on: 1530122

The bug 1525274 is a blocker for my add-on (I seem to not have the permissions to set it as a blocker for this bug)

Depends on: 1525274
Depends on: 1542643
Depends on: 1529785

(In reply to public from comment #1)

Only then can uBlock Origin be supported for Thunderbird:
https://github.com/gorhill/uBlock/issues/117

uBlock Origin issue merged to https://github.com/gorhill/uBlock/issues/3698

Depends on: 1546915
Depends on: 1550960

Not quite sure where to put this... but are there any plans to expose the email address in browser.accounts.list()?

I'm writing an extension which synchronizes email client tags (in a way some imap servers don't allow), and namespacing the synced data by email address would be helpful.

Not quite sure if this is the right place to put a comment like this though.

Not directly, no, because an account can have more than one email address associated with it. Bug 1531593 is the one to follow, but as you can see, there's no progress yet.

Depends on: 1575693
Depends on: 1575708
Depends on: 1575710
Depends on: 1532528
Depends on: 1587673
Depends on: 1585995
Depends on: 1595824
Depends on: 1598078
Depends on: 1598332
Depends on: 1599380
Depends on: 1606185
Depends on: 1607192
Depends on: 1611700
Depends on: 1613534
Depends on: 1613535
Depends on: 1613536
Depends on: 1613562
Depends on: 1616114
Depends on: 1603489
Depends on: 1617758
Depends on: 1618828
Depends on: 1619057
Depends on: 1619416
Depends on: 1620611
Depends on: 1615801
Depends on: 1621562
Depends on: 1622908
Depends on: 1623704
Depends on: 1624059
Depends on: 1624136
Depends on: 1624836
Depends on: 1625746
Depends on: 1625780
Depends on: 1627197
Depends on: 1627200
Depends on: 1627203
Depends on: 1627205
Depends on: 1627315
Depends on: 1627472
Depends on: 1627556
Depends on: 1627604
No longer depends on: 1627556
No longer depends on: 1627200
No longer depends on: 1627197

From this point on I am not using this bug. It was intended as a meta-bug for feature requests, but it has become a magnet for anything to do with WebExtensions, which is unnecessary because that's what the "Add-Ons: Extensions API" category is all about.

Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED

Status from https://developer.thunderbird.net/add-ons/about-add-ons#extensions

  • MailExtension: The only valid extension type in Thunderbird 74 and later.
  • Legacy WebExtensions: Deprecated in Thunderbird 74.
  • Legacy Bootstrap Extension: Deprecated in Thunderbird 68.
  • Legacy Overlay Extension: Deprecated in Thunderbird 68.

Current releases and release notes:

Release schedule:

Blocks: 1679818
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: