User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:69.0) Gecko/20100101 Firefox/69.0

Steps to reproduce:

Created an add-on that opens a web page (url) in a new tab, via browser.tabs.create(). The tab opens and the page is displayed correctly. But clicking on any link in the page does not display the new page in the same tab, it always opens the new page in the system default browser app. This happens even if the link's anchor tag contains 'target="_self" rel="noopener noreferrer"'.

Expected results:

Please make 'target="_self" rel="noopener noreferrer"' in an anchor tag work correctly in an anchor tag in a web page in a Thunderbird tab: new page should open in same tab, inside TB.

Please add a preference to control whether anchor links without "target" attribute open new page in same Thunderbird tab, or in system browser.

Forgot to specify I'm using TB 70.0b2 (64 bit).

You don't really need an add-on, you can issue openContentTab("http://www.google.com", "tab"); in the error console. But all the links on the page will open in the external browser.

Alta88, you had an add-on to open web pages in TB. Is there a trick to open links also in TB?

Yep, there sure are tricks. And they're tricky. You can look at TotalMessage extension below, it even implements back/forward in a Tb content tab. And no, you can't do back/forward in a web extension unless it's in an experiment.
https://bitbucket.org/alta8888/totalmessage/wiki/Home

The real solution is to expose/alter the web extension api so it either lets the caller configure a clickhandler regex or allows all by default.

A quick look at TotalMessage's code gives me the impression it's catching all mouse-click and key-press events, and essentially re-implementing normal browser handling of links. Not sure.

Yes, I think the real solution is to fix/change how TB's handling of links in web pages works. Make 'target="_self"' work, and add a preference to control whether anchor links without "target" attribute open new page in same Thunderbird tab, or in system browser.

It does capture events to be keyboard aware, but that's not relevant to link clicks. You need to look for the arguments of openTab() there or in the comm source and have a clickhandler. And no, the target stuff is not the solution.

I agree that forcing web page anchors to contain a "target" attribute is not the solution, but if it's present then target="_self" should work correctly, not open another tab/window (in a completely different app, too!).

I think TB should add a preference to control whether anchor links without "target" attribute open a new page in same Thunderbird tab, or in system browser.

I think forcing add-ons to implement clickhandlers to make web page anchor tags work reasonably would be a bad solution.

You really need to look at the what the argument does: it's a somewhat lame attempt to prevent 'browsing' in Tb and works by limiting what urls are opened internally by doing string compares on the link url based on the arg regex (misnamed as clickhandler), as I've already said. And certainly my extension lets you browse any link in a Tb content tab, including back/forward, and neither excludes nor handles specially any anchor links, so I don't see your point. The only issue is that openTab() args are abstracted away and it's not a web extension api.

Sorry, I wasn't talking about your extension, I'm talking about the general case, even when a web page has been opened in TB with no add-ons installed (if that's possible). The default case in TB with no add-ons should be that a link with target="_self" works as expected, and there should be a preference to control how links with no "target" attribute work.

With no addons, how exactly will you open a web page in a content tab? Are you talking about feed web pages in a tab - that is the only way to get to a non Tb related web page in a content tab. As I've said, 'browsing' support in Tb is intentionally (incorrectly imo, which is why I made an extension) limited.

If, instead, you really are referring to your extension's web page, what happens to your anchor links if you issue this in console, replacing the url with some page with anchor links:

openTab("contentTab", { contentPage: "https://mozilla.com", clickHandler: "specialTabs.siteClickHandler(event, new RegExp(\"^\"));", background: true }, "tab");

I don't know much about TB, maybe there's no way to open a web page without an add-on. Click a link in an email ? Someone said do "open" in the inspector/debugger, but that's cheating.

Why is browsing in TB limited ? I could see that maybe complicated features might be too much trouble to support, but surely basic links should work with no trickiness.

I don't think add-ons should have to supply click-handlers to make web pages work.

What I want to do is make an add-on that brings up the main web page of some messaging site such as WhatsApp for which there is no integration. That web page should work as normally as possible inside TB, as well as it works outside in a browser. Maybe my TB add-on could scrape data from WhatsApp pages, to replicate the user's WhatsApp contacts into TB's contacts.

'browsing' support in Tb is intentionally limited.

Can we discuss this policy, please, and maybe get it changed ? Why is browsing limited, to the point where clicking on a link doesn't work properly (IMO) ?

Thunderbird is not designed to be a browser; to make it so would mean importing a lot of code from an actual browser. However, comfortable ergonomics and partial browsing features are useful, so you can try this extension to open links in and browse in a content tab.
https://bitbucket.org/alta8888/browseintab

(In reply to alta88 from comment #12)

https://bitbucket.org/alta8888/browseintab

Thanks ! That does much of what I want. But I would want a link-click in a page to open the new page in the same tab, not a new tab. And the site I want to use, web.whatsapp.com, objects to the user-agent string, so I'd need to set that in a preference. If you're actively developing this add-on, would you be willing to add those features ? Or I could clone the code and make my own version. Thanks !

1. In a content tab, if the web page decides to add target=_blank, it wants to open in another tab. In firefox, there isn't anything native to do about it, that's what's in the spec. However, this can be annoying and the user should override whatever they want, so I can add this as a UI option. If you mean from somewhere other than a content tab, like messagepane, then that would mean a hack of basic Tb tabbing "logic". Duplicate url clicks make the same tab focus, but otherwise it's always a new tab.

2. You can always add general.useragent.override with whatever you want. There is a bool, compatMode.firefox, that should also work. Except that it demonstrates a whatsapp ua parsing fail - it sees firefox but can't get the right version. I will add a UI option for using the compat string for opening links only, there can be side effects to making it permanent for all web traffic. I suggest you ask whatsapp to fix their ua regex rather than hardcoding the pref though. Firefox has a web compat team for things like this, maybe this is a service a community manager person should do for Tb.

For item 1, I was talking about a normal link in a web page, with no "target" specified. In Firefox, the page linked to is opened in the same tab. In TB, I think it should be the same. I'm talking only about web pages, not messages.

(In reply to alta88 from comment #3)

....
The real solution is to expose/alter the web extension api so it either lets the caller configure a clickhandler regex or allows all by default.

Is this something for Thunderbird? or for Core?

(In reply to Bill Dietrich from comment #11)

'browsing' support in Tb is intentionally limited.

Can we discuss this policy, please, and maybe get it changed ? Why is browsing limited, to the point where clicking on a link doesn't work properly (IMO) ?

(In reply to alta88 from comment #12)

Thunderbird is not designed to be a browser; to make it so would mean importing a lot of code from an actual browser. However, comfortable ergonomics and partial browsing features are useful, so you can try this extension to open links in and browse in a content tab.
https://bitbucket.org/alta8888/browseintab

Just to clarify this a bit more: When we first implemented the ability for viewing pages in tabs for Thunderbird, we wanted to make it so that to view what's new, or add-ons you could do that within Thunderbird, to avoid the context switch out to a different application. We did decide that when you then start actually browsing to other places, you should go to the browser - we didn't want to reproduce all the address bar, back/forward & other functionality for obvious reasons, we did want to make it so that you could do some things within TB (e.g. easily install add-ons).

One of the other thoughts at that time was that add-ons could produce that functionality, and also provide the address bar etc.

(In reply to Wayne Mery (:wsmwk) from comment #16)

(In reply to alta88 from comment #3)

The real solution is to expose/alter the web extension api so it either lets the caller configure a clickhandler regex or allows all by default.

Is this something for Thunderbird? or for Core?

Definitely Thunderbird.

(In reply to Mark Banner (:standard8) (afk until 13 Aug) from comment #17)

(In reply to Bill Dietrich from comment #11)

'browsing' support in Tb is intentionally limited.

Can we discuss this policy, please, and maybe get it changed ? Why is browsing limited, to the point where clicking on a link doesn't work properly (IMO) ?

(In reply to alta88 from comment #12)

Thunderbird is not designed to be a browser; to make it so would mean importing a lot of code from an actual browser. However, comfortable ergonomics and partial browsing features are useful, so you can try this extension to open links in and browse in a content tab.
https://bitbucket.org/alta8888/browseintab

Just to clarify this a bit more: When we first implemented the ability for viewing pages in tabs for Thunderbird, we wanted to make it so that to view what's new, or add-ons you could do that within Thunderbird, to avoid the context switch out to a different application. We did decide that when you then start actually browsing to other places, you should go to the browser - we didn't want to reproduce all the address bar, back/forward & other functionality for obvious reasons, we did want to make it so that you could do some things within TB (e.g. easily install add-ons).

Not sure what obvious reasons would justify preventing uncomfortable context switches. Sometime a fast look at a link in a foreground tab is most workflow efficient. It's not like that and b/f navigation is hard.

One of the other thoughts at that time was that add-ons could produce that functionality, and also provide the address bar etc.

A somewhat enhanced address bar is implemented in BrowseInTab. It's not intended to be writeable, although it could do that. The existing b/f buttons don't have a very quality history UX, but that could be added too. Tab session history is not likely and a full Firefox like urlbar and bookmarks are not happening -- those are things for a real browser. However, the latest version does implement Firefox like icons for various page types, a refresh button, and even a per tab persisted zoom level indicator. A few more Firefox like decoration could be added.

(In reply to Wayne Mery (:wsmwk) from comment #16)

(In reply to alta88 from comment #3)

The real solution is to expose/alter the web extension api so it either lets the caller configure a clickhandler regex or allows all by default.

Is this something for Thunderbird? or for Core?

Definitely Thunderbird.

Yes, core doesn't have a <browser> in tabs problem.

I do not know if the default behavior of clicks in a tab will be changed, but this bug is filed in the add-ons sections and with your own add-on, you can provide a content_script, which fixes this. I showed a solution here: https://bugzilla.mozilla.org/show_bug.cgi?id=1618828#c3

This works for basic WebExtensions and back and forth worked for me as well. Does that satisfy the needs of comment 0 ?

This bug was filed in the Add-on section, but add-ons can change the click behavior via content scripts. It is therefore invalid here. Moving this to general, as it looks like the bug creator wants to have the general behavior changed. See reproduction steps in comment 2.

I think the default click handler should have a reasonable implementation. Then, the addon should be able to easily override it, if it wants something different, but the default should be sane.

We have 3 differenmt locations:

• browserAction popup, options page, and similar addon UI
• browser tab in Thunderbird
• browser window in Thunderbird

and 3 possible behaviors:

• open in same place
• open in new tab or window (depending on where the link is) within Thunderbird
• open the systen browser

If we're in a browserAction popup or similar, with an addon URL (chrome or addon URL or similar), and the link target is https: or http:, then a <a href="https://...> should open a tab or window or the system browser.

If we're already in a tab, and an a linke has a target=_blank, i.e. <a href="https://..." target="_blank">, then it should open a new tab. Likewise, browser windows should behave the same.

In any case, the page author should be able to control the behavior using the <a target=...> attribute. He shouldn't need a script for that, but be able to use the standard HTML markup to control the behavior, if the default doesn't fit his case. We could use _self to replace the current page in-place, _blank to mean new tab, browser window or system browser, depending on circumstance, and tab to mean a new tab within Thunderbird, and another keyword to mean the system browser.

As the default click handler in Thunderbird web tabs is forcing all links to be opened externally, it also breaks custom defined protocol handlers. They work as expected in WebExtension windows.

(In reply to John Bieling (:TbSync) from comment #22)

As the default click handler in Thunderbird web tabs is forcing all links to be opened externally, it also breaks custom defined protocol handlers.

Is this something we still support going forward?

This has been inconsistent for quite a while. The expected behaviour is that same-site links are opened in Thunderbird, while all other links are opened externally. Standard8 gave a good explanation why we do it this way. The attached patch returns to this behaviour and locks it down with a test.

Add-ons can either open pages directly in the user's default browser (windows.openInDefaultBrowser) or in a tab (tabs.create). If a remote page was deliberately opened inside Thunderbird, then its same-site links will also open in Thunderbird to respect that choice, but if the user then browses away to other sites, these pages will be opened in the browser again.

The target attributes do not change this. What they change is whether same-site links are opened in the same tab or in a new tab. Please note that targets other then "_self" are currently broken badly (Bug 1770105).

If the add-on really wants to override the same-site behaviour, it must register a content script and capture click events and manually handle them. This patch is adding this information to the documentation of tabs.create.

The broken protocol_handlers are being fixed in Bug 1757142.

Depends on D167661

The second patch restores the correct behaviour for popup windows as well. Same-site links are opened in the same popup, everything else is opened in the user's browser. Same-site links with a target other than _self are opened in a new tab in the most recent normal Thunderbird window, which is what Firefox is doing in this case as well.

Tree is a bit orange at the moment. Without me:
https://treeherder.mozilla.org/jobs?repo=try-comm-central&revision=4c0d2b7ee1e85b5023828b9c82346bd3a9aa375e

normal:
toolkit/components/extensions/test/xpcshell/test_ext_contentscript_errors.js
toolkit/profile/xpcshell/test_snap.js
toolkit/profile/xpcshell/test_snap_empty.js
comm/mail/components/extensions/test/browser/browser_ext_contentScripts.js | backgroundColor
comm/mail/components/extensions/test/browser/browser_ext_contentScripts.js | textContent
comm/mail/test/browser/content-policy/browser_generalContentPolicy.js
comm/calendar/test/browser/invitations/browser_imipBarExceptionOnly.js

debug:
comm/mail/components/extensions/test/browser/browser_ext_commands_execute_message_display_action.js | application terminated with exit code 11
comm/mail/components/extensions/test/browser/browser_ext_menus_replace_menu.js
xpcshell.ini:comm/mailnews/imap/test/unit/test_saveTemplate.js
xpcshell_maildir-cpp.ini:comm/mailnews/imap/test/unit/test_imapStatusCloseDBs.js
xpcshell-cpp.ini:comm/mailnews/imap/test/unit/test_imapStatusCloseDBs.js
xpcshell-remote.ini:toolkit/components/extensions/test/xpcshell/test_ext_userScripts_register.js
toolkit/components/backgroundtasks/tests/xpcshell/test_backgroundtask_no_output.js
comm/calendar/test/unit/test_invitationutils.js
/builds/worker/checkouts/gecko/xpcom/build/LateWriteChecks.cpp:120
comm/mail/test/browser/attachment/browser_openAttachment.js

I see ne additional fails in this try run:
https://treeherder.mozilla.org/jobs?repo=try-comm-central&revision=c88464dce45199a9f1537e5441eb808be4130289

Pushed by geoff@darktrojan.net:
https://hg.mozilla.org/comm-central/rev/d0960e7180cf
Fix default click handler for WebExtensions. r=darktrojan

Pushed by geoff@darktrojan.net:
https://hg.mozilla.org/comm-central/rev/73afd504889f
Fix default click handler for WebExtensions, part 2. r=darktrojan