User Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.96 Safari/537.36

Steps to reproduce:

Setup an email account with O365 that has Microsoft's 2FA setup and does not allow Application Passwords (b/c it's not available for all integrations with O365).

Actual results:

Authentication Denied as there is no ability to either pull a token from Firefox/Chrome, or perform the 2FA authentication.

NOTE: From my understanding in various Thunderbird plugin discussions (such as https://github.com/ExchangeCalendar/exchangecalendar/issues/47) having a plug-in handle the authentication is not possible at least under the current design.

Expected results:

Dialog pop-up to interact with O365 in order to perform the 2FA authentication and get a token that Thunderbird can then re-use for the duration of the session. This should be doable via either Thunderbird directly or via an account authentication plug-in infrastructure.

NOTE: Due to Microsoft's pricing structure and features the Application Password functionality is not available to every O365 integration/deployment. A work around is for companies to allow fall-back to non-2FA authentication but that is not very desirable and requires companies to make special arrangements for subsets of users to do this.

NOTE: A solution where plug-in authors could provide authentication functionalities to support this would be excellent and would seem to be a proper solution to the issues raised in related discussions in https://bugzilla.mozilla.org/show_bug.cgi?id=1427538.

https://bugzilla.mozilla.org/show_bug.cgi?id=1427538 points out the OAuth functionality; however, Microsoft doesn't use that. They have their own 2FA protocol. Sadly it's another case of Microsoft doing their own thing versus using the standards already available. Thus I think this is a different (but related) but to https://bugzilla.mozilla.org/show_bug.cgi?id=1427538.

Application passwords may not always work, but then I think POP3 and IMAP are also disabled and you need another protocol (MAPI, OWA?) to access your e-mail, which isn't supported yet by Thunderbird. Please share some more information on your situation: how do access outlook from Thunderbird, which authenticator app do you use, have you tried setting up your device as a trusted device?

Sorry for the delay....

So the basic situation was that O365 Exchange had IMAP/SMTP enabled for use by Thunderbird, just like anything else; MAPI/OWA protocols were required. However, the organization wanted to use 2 Factor Authentication but did not have a configuration that allowed for using O365's Application Password capability (not available for every O365 integration)

It would have been really nice to have Thunderbird support the Microsoft 2FA Protocol (aka Modern Auth) directly which would have allowed things to just work. Other application (Slack Desktop, CryptZone AppGate) due so via a browser integration so that tokens can be re-used and the browser can manage the auth-session and thus maintain an SSO solution. Whether Thunderbird does the browser integration method or directly auths against O365 doesn't really matter to me; I'm just looking for a better authentication experience against O365 for services Thunderbird already supports, and extensions (f.e ExchangeCalendar) take advantage of.

I'm not asking for native Outlook (MAPI/OWA) functionalities be integrated; just Modern Auth capabilities. This would also help folks with convincing their organizations to enable IMAP/POP3 too.

I was fortunate in that we had a non-insignificant group of folks using IMAP/SMTP so they worked out a solution via VPN. It would just be really awesome to not have to do work arounds especially as more and more folks are moving towards O365 and 2FA.

Perhaps a good solution may be to create a plug-in interface for authentication so that extensions could be written to support various kinds of authentication, including the variety 2FA methods out there? Just a thought.

To more directly answer the questions:

"Application passwords may not always work, but then I think POP3 and IMAP are also disabled and you need another protocol (MAPI, OWA?) to access your e-mail, which isn't supported yet by Thunderbird."

Correct - Application Passwords are not always architecturally feasible with how ADFS works. Some integrations don't allow them; but they're also an add-on cost IIRC. Organizations can, however, choose whether or not to enable POP3 and IMAP even with O365. The key factor comes down to authentication which is what this request is about.

"How do access outlook from Thunderbird?"

Access is IMAP+SMTP and ExchangeCalendar with Lightning. Nothing fancy, and no MAPI/OWA stuff in play.

"Which authenticator app do you use?"

A solution was worked out to continue allowing use of the current options in Thunderbird, but required being on VPN to do so. The organization preferred to use the Microsoft 2FA (Modern Auth) provided by O365. If VPN was down, then 2FA was required for security purposes. It would be great to be able to simply use 2FA and not be required to be on VPN.

"Have you tried setting up your device as a trusted device?"

I'm on a Linux laptop (Kubuntu) and don't integrate into AD/ADFS - no need for SaMBa, etc. So no that's not a possible solution.

In https://mzl.la/2FqbQj0 are 2-3 related bug reports

Im very interesting in this topic, if 2FA is better supported by thunderbird, it will increase alot the usability of Thunderbird with products like Outlook365.

How does this enhancement request relate to:

https://bugzilla.mozilla.org/show_bug.cgi?id=1330049

So, you duped my bug 1330049 to this one, but...

Are you saying that OAUTH2 is the same thing as 2FA?

As far as I know, they are completely different - but maybe if this one is fixed, it will also provide OAUTH2 support?

If this is implemented as I described in one of my last comments then it would provide a generic framework within which OAUTH2 support can be implemented, as well as Microsoft Modern Auth and other auth systems. I'm okay with this morphing into a generic auth plugin enhancement as I think through the conversation that is what has happened.

This might be simple to do by adding relevant account information to https://searchfox.org/comm-central/source/mailnews/base/util/OAuth2Providers.jsm . Thunderbird would need to register for an Azure account, and we need to make sure we don't have any restrictions. I'm running a build that attempts to do that. It could be that I have the scopes wrong, it seems I'd really need wl.imap and wl.offline_access but those are deprecated.

I'm not sure though how it would work for domains that are not office365.com, e.g. enterprise domains. For gmail, enterprise users can just sign in via smtp.gmail.com

I've been able to log in to an outlook.com account using OAuth2 with steps as described. I'm going to need a test account for whatever enterprise thing you have to see if I can get it to work there.

@bm_witness: excellent, works for me...

@Phillip: we use Office365 with our own domain. When setting up Outlook, it does everything for you and everything (the settings) are all hidden.

But I still use Thunderbird, and use mail.office365.com and smtp.office365.com for the server settings.

I'd be happy to set you up with a test account if it will help, just contact me privately...

Done, I've sent you an email. Glad to hear that outlook is using the main server settings as well.

A few things to consider:

• In Thunderbird, use OAuth2 only when it's necessary. It gives a poor customer experience, because OAuth expires and the user has to re-login, at unknown intervals, it's completely up to the server, whether it's 3 hours or 3 months. We cannot save the password. One of the primary reasons why people want to use Thunderbird instead of Outlook Web Access is because they don't want to re-login, but be automatically logged in. So, long story short: Do not enable OAuth2 for all users, but only those where it's absolutely necessary. In Owl, we check at setup which authentication is available for a given account, and prefer automatic background logins without webpage wherever possible.

• Office365 supports custom authentication, with login pages hosted at the premises of the enterprise. That might be Microsoft's auth server, or a fully custom page. So, even if it works in the cases you tested, it doesn't mean it will work for other users.

• There are tons and tons of variations. In fact, supporting the login pages so that it works for all users was a royal pain and took several weeks of development. It worked for us, but not for some of our customers. You can imagine how difficult that was.

This is not just a question of putting some config data in somewhere. You might break more users than you fix.

(In reply to Philipp Kewisch [:Fallen] [:📆] from comment #11)

This might be simple to do by adding relevant account information to https://searchfox.org/comm-central/source/mailnews/base/util/OAuth2Providers.jsm . Thunderbird would need to register for an Azure account, and we need to make sure we don't have any restrictions. I'm running a build that attempts to do that. It could be that I have the scopes wrong, it seems I'd really need wl.imap and wl.offline_access but those are deprecated.

I'm not sure though how it would work for domains that are not office365.com, e.g. enterprise domains. For gmail, enterprise users can just sign in via smtp.gmail.com

Keep in mind MS uses Modern Auth too, which is what my original request was about. By having a framework for auth plugins we can offload this from TB Core and let the community help out more in ways they cannot now. Then you can get OAUTH, OAUTH2, Modern Auth, and whatever comes next. The initial modules should be the current auth methods.

@Phillip:
I just wanted to correct the record for this bug - I have no idea what Outlook uses, since they hide the actual server settings. An Enterprise customer sets up their auto-discover DNS records, and Outlook just magically sets everything up. If someone knows how to peek at the actual server settings being used, and wants me to check, let me know where to find them.

@Ben:
I've been using OAUTH2 with GMail for ... well, I'm not sure, but I'm pretty sure its been well over a year, and never had to re-login. So I guess this is server dependent (only a problem with Office365)?

Also, I'm talking about using Thunderbird vs Outlook, not OWA, two totally different beasts.

Of course, Microsoft has always been more than happy to break things for third parties using their stuff without having a licensing deal in place, and there's likely not much if anything we can do about that.

In my Office365 enterprise account with custom login, I'm logged out every few hours. It's highly annoying.

Microsoft has always been more than happy to break things for third parties using their stuff

Yes. That's precisely my point.

However, as long as we're using the standard IMAP, they have a very hard case to make to break a standard protocol and all users that use it. The more users are already using OAuth instead of standard IMAP auth, the more easily they can break standards.

So, moving to OAuth actually puts us at higher risk of being broken than staying with the standard IMAP protocol and its login mechanisms.

More generally, what we need is an automatic login, and for it to work always, every time, and without a sudden "used to work, but doesn't work anymore" at an undefined later time. OAuth2 makes no guarantees at all https://hueniverse.com/oauth-2-0-and-the-road-to-hell-8eec45921529 .

What I'm saying is: Do not use OAuth2 for users that can use standard IMAP login.

(In reply to Ben Bucksch (:BenB) from comment #18)

In my Office365 enterprise account with custom login, I'm logged out every few hours. It's highly annoying.

I'm guessing you mean using OWA? I only use that occasionally, and never for more than just doing one off tasks.

I just checked, and apparently there is an organization timeout (default is 6 hours) that controls this that can be changed by an Admin using powershell:

ActivityBasedAuthenticationTimeOutInterval

From what I see it is not a per user setting, and no way to change it in the GUI.

A workaround (untested/unconfirmed) is to use an Addon that periodically refreshes the browser.

Also - IMAP isn't a guaranteed 'will always work' either. IMAP access on Office365 accounts is disabled by default (at least it is on ours).

So the issue is that many users don't have a choice. They are required to use a 2FA auth method and not all are compatible with TB right now. TB needs to give users the ability to have more auth options as required by their organizations or users will go away.

My former employer required Modern Auth with O365. Nothing I could do. Managed to get a special exception when on VPN, but no guarantee how long such an exception would stick around, let alone how easy it is for others to get similar set ups approved and implemented. The more hurdles the less likely they'll be able to use TB.

In the end, IMAP auth is going the way of the DoDo, even for most providers. Google and Yahok has a special keys to use, and any enterprise will require multi-factor.

So the issue is that many users don't have a choice

FWIW, I am talking about users who do have a choice to use IMAP. We should keep them on IMAP with normal IMAP or SASL authentication methods, not OAuth2. That's all I meant.

They are required to use a 2FA auth method and not all are compatible with TB right now

FWIW, you can use Owl. It works with 2FA. Disclaimer: Owl is my product.
https://addons.thunderbird.net/thunderbird/addon/owl-for-exchange

Fwiw, I just want to use TB and select the right auth method. Not interested in OWA and that would still require Modern Auth support. Further I also use Lightening+ExchangeCalendar which is why I'm interested in a framework solution because the add-ons cannot easily use each either that way right now.

With my proposal if a user wanted to use IMAP auth they still could, they'd just also have more options for when it's not possible. And you're add-on could be simplified by not having conceal with all the auth stuff too.

FYI this becomes more urgent to address now as MS have announced retirement of Basic auth over IMAP/POP for O365.

I wasn't able to get this to work with the enterprise account, and my understanding from Microsoft is that XOAUTH isn't supported for enterprise configurations. We can push the patches here to make it work for outlook.com, but that won't fix the enterprise case.

Let's keep the comments on topic here please, this bug should be focused on implementing the feature.

We're expecting to get this fixed in the next two months or so.

I'm confused by the fact that outlook.office365.com:993 advertises the AUTH=XOAUTH2 capability but Thunderbird doesn't seem to try using it (even when I manually select OAuth in my server settings).

[(null) 4570: Unnamed thread 0x784ace6575e0]: I/IMAP 0x784acb7c0000:outlook.office365.com:NA:CreateNewLineFromSocket: * CAPABILITY IMAP4 IMAP4rev1 AUTH=PLAIN AUTH=XOAUTH2 SASL-IR UIDPLUS ID UNSELECT CHILDREN IDLE NAMESPACE LITERAL+
[(null) 4570: Unnamed thread 0x784ace6575e0]: D/IMAP ReadNextLine [stream=0x784ac85df100 nb=28 needmore=0]
[(null) 4570: Unnamed thread 0x784ace6575e0]: I/IMAP 0x784acb7c0000:outlook.office365.com:NA:CreateNewLineFromSocket: 1 OK CAPABILITY completed.
[(null) 4570: Unnamed thread 0x784ace6575e0]: D/IMAP try to log in
[(null) 4570: Unnamed thread 0x784ace6575e0]: D/IMAP IMAP auth: server caps 0x800087635, pref 0x0, failed 0x0, avail caps 0x0
[(null) 4570: Unnamed thread 0x784ace6575e0]: D/IMAP (GSSAPI = 0x1000000, CRAM = 0x20000, NTLM = 0x100000, MSN = 0x200000, PLAIN = 0x1000, LOGIN = 0x2, old-style IMAP login = 0x4, auth external IMAP login = 0x20000000, OAUTH2 = 0x800000000)
[(null) 4570: Unnamed thread 0x784ace6575e0]: D/IMAP no remaining auth method
[(null) 4570: Unnamed thread 0x784ace6575e0]: E/IMAP login failed entirely

Happy to provide more details if you need them.

@(In reply to Jesse Thompson from comment #33)

I'm confused by the fact that outlook.office365.com:993 advertises the AUTH=XOAUTH2 capability but Thunderbird doesn't seem to try using it (even when I manually select OAuth in my server settings).

[(null) 4570: Unnamed thread 0x784ace6575e0]: I/IMAP 0x784acb7c0000:outlook.office365.com:NA:CreateNewLineFromSocket: * CAPABILITY IMAP4 IMAP4rev1 AUTH=PLAIN AUTH=XOAUTH2 SASL-IR UIDPLUS ID UNSELECT CHILDREN IDLE NAMESPACE LITERAL+
[(null) 4570: Unnamed thread 0x784ace6575e0]: D/IMAP ReadNextLine [stream=0x784ac85df100 nb=28 needmore=0]
[(null) 4570: Unnamed thread 0x784ace6575e0]: I/IMAP 0x784acb7c0000:outlook.office365.com:NA:CreateNewLineFromSocket: 1 OK CAPABILITY completed.
[(null) 4570: Unnamed thread 0x784ace6575e0]: D/IMAP try to log in
[(null) 4570: Unnamed thread 0x784ace6575e0]: D/IMAP IMAP auth: server caps 0x800087635, pref 0x0, failed 0x0, avail caps 0x0
[(null) 4570: Unnamed thread 0x784ace6575e0]: D/IMAP (GSSAPI = 0x1000000, CRAM = 0x20000, NTLM = 0x100000, MSN = 0x200000, PLAIN = 0x1000, LOGIN = 0x2, old-style IMAP login = 0x4, auth external IMAP login = 0x20000000, OAUTH2 = 0x800000000)
[(null) 4570: Unnamed thread 0x784ace6575e0]: D/IMAP no remaining auth method
[(null) 4570: Unnamed thread 0x784ace6575e0]: E/IMAP login failed entirely

Happy to provide more details if you need them.

This isn't simply about OAUTH support. Microsoft actually implements another auth system called Modern Auth, which was what initiated this bug report. Support for OAUTH, JWT, SAML, and other mechanisms is needed which is why this turned into a call for a plugin/extension framework for authentication so the community can provide all the tools needed for any random auth that might pop up instead of requiring the Thunderbird itself to be updated to support it.

Oh - per O365 and OAUTH, Basic Auth, Modern Auth - it all depends on what the organization you're part of chooses to provide to you.

Microsoft is planning to retire basic auth in October 2020 along with a promise to add OAuth 2.0 support to Office 365's IMAP before then. When we talked to them at Ignite they said that the feature will be released very soon.

Yes, this is all part of their Modern Auth capabilities/strategy, which involves OAuth2, but I doubt anyone knows what the detailed IMAP implementation will look like.

I was speculating (hoping) that the existence of XOAUTH2 in their IMAP CAPABILITY string was an indication that they silently released it. I interpret what you're saying is that Thunderbird did another step to validate the XOAUTH2 advertised capability and determined that it isn't yet functional. I can't seem to find the logs or code that shows where it might be failing.

The idea that this is dependent on the organization (Microsoft calls them "tenant"s) doesn't make sense. Wouldn't the user need to authenticate somehow (via OAuth?) to indicate to the server what tenant the user is part of?

Since this is a pre-authentication decision, I think that the client would need to first try OAuth (since it's advertised) and then fall back to basic auth only if OAuth fails. Maybe that's what is happening now, for all I can tell from the Thunderbird debug logs.

(In reply to Jesse Thompson from comment #36)

Microsoft is planning to retire basic auth in October 2020 along with a promise to add OAuth 2.0 support to Office 365's IMAP before then. When we talked to them at Ignite they said that the feature will be released very soon.

Yes, this is all part of their Modern Auth capabilities/strategy, which involves OAuth2, but I doubt anyone knows what the detailed IMAP implementation will look like.

I was speculating (hoping) that the existence of XOAUTH2 in their IMAP CAPABILITY string was an indication that they silently released it. I interpret what you're saying is that Thunderbird did another step to validate the XOAUTH2 advertised capability and determined that it isn't yet functional. I can't seem to find the logs or code that shows where it might be failing.

Interesting. I don't know all Thunderbird does for validating XOAUTH2, just know what my former employer did and how the settings roll out. For example, Microsoft has the ability to do Application Passwords/Keys, but only if the org pays extra for it. There's numerous things that determine whether or not you need to do Modern Auth or if there's another auth mechanism you can use and it's all org dependent. It's also dependent on if you're doing an AD or ADFS integration among other things.

(In reply to Jesse Thompson from comment #37)

The idea that this is dependent on the organization (Microsoft calls them "tenant"s) doesn't make sense. Wouldn't the user need to authenticate somehow (via OAuth?) to indicate to the server what tenant the user is part of?

Since this is a pre-authentication decision, I think that the client would need to first try OAuth (since it's advertised) and then fall back to basic auth only if OAuth fails. Maybe that's what is happening now, for all I can tell from the Thunderbird debug logs.

Based on previous experience, Microsoft takes the username (email - domain will determine tenant through the AD/ADFS relationships) and then figures out the organization and applies org specific settings. It's a multi-step process in the web-browser - first identify your user so it knows where to send you, then apply the 2FA process based on your orgs functionality; of course the web-browser stuff is more SAML based so not sure how all it's functional in the application auth flow, but I imagine not too dissimilar. Other applications can also integrate with a web-browser to use the same token too so you can do a SSO kind of login with it; or you can go an implement the Modern Auth protocol for direct authentication (they've got a JS library and documentation).

All-in-all, I think there's enough various options that people really need a choice. And when the next thing comes along then we need a faster way to respond with new functionality too - so a pluggable auth system makes very good sense. Waiting to see what the TB devs are thinking given Comment #32.

(In reply to bm_witness from comment #38)

All-in-all, I think there's enough various options that people really need a choice. And when the next thing comes along then we need a faster way to respond with new functionality too - so a pluggable auth system makes very good sense. Waiting to see what the TB devs are thinking given Comment #32.

O365 users (and their tenant admins) won't have a choice in October, since IMAP basic auth is being completely retired (assuming Microsoft doesn't change their plan)

Thank you for your attention to this issue. We will have a lot of happy Thunderbird O365 users if this works out!

Gnome's Evolution Email program on Linux does seem to support Microsoft's oauth2 with modern authentication, though we did have to create an application to make this work.
This might be helpful for anyone looking at an implementation to copy:

https://wiki.gnome.org/Apps/Evolution/EWS/OAuth2

(In reply to buildtherobots from comment #40)

Gnome's Evolution Email program on Linux does seem to support Microsoft's oauth2 with modern authentication, though we did have to create an application to make this work.
This might be helpful for anyone looking at an implementation to copy:

https://wiki.gnome.org/Apps/Evolution/EWS/OAuth2

Is this code GPL, only, like the rest of Gnome? Then we wouldn't be allowed to copy it directly, but maybe it could help as inspiration.

(In reply to buildtherobots from comment #40)

Gnome's Evolution Email program on Linux does seem to support Microsoft's oauth2 with modern authentication, though we did have to create an application to make this work.
This might be helpful for anyone looking at an implementation to copy:

https://wiki.gnome.org/Apps/Evolution/EWS/OAuth2

Evolution is using EWS (Exchange Web Services) which is an HTTP-based protocol. Thunderbird already has an add-on that works(?) with EWS (ExQuilla)

My understanding is that this issue has to do with making Microsoft's OAuth 2.0 IMAP implementation work natively with Thunderbird (without requiring an add-on). A commitment to native support for open standards seems like a worthy goal for Thunderbird.

I assume that Microsoft's OAuth 2.0 implementation within IMAP is different than EWS; most notably the fact that the former hasn't been confirmed to actually work yet.

(By the way, any developer who needs information about obtaining an O365 account for testing purposes please contact me directly)

(In reply to Jesse Thompson from comment #42)

(In reply to buildtherobots from comment #40)

Gnome's Evolution Email program on Linux does seem to support Microsoft's oauth2 with modern authentication, though we did have to create an application to make this work.
This might be helpful for anyone looking at an implementation to copy:

https://wiki.gnome.org/Apps/Evolution/EWS/OAuth2

Evolution is using EWS (Exchange Web Services) which is an HTTP-based protocol. Thunderbird already has an add-on that works(?) with EWS (ExQuilla)

My understanding is that this issue has to do with making Microsoft's OAuth 2.0 IMAP implementation work natively with Thunderbird (without requiring an add-on). A commitment to native support for open standards seems like a worthy goal for Thunderbird.

Yes and no. Yes making Microsoft's Modern Auth work natively in Thunderbird, but doing so via a new plugin architecture so that the community can help support the authentication mechanisms going forward instead of having to wait for years to get it done. I realize that the TB devs time is limited so they have to prioritize work; thus a community approach should help, even if the allowed plugins have to go through some kind of validation process before being accepted to the plugin site to help prevent malicious plugins.

When I could I was using the ExchangeCalendar plugin to get Lightning and Exchange working together; however, the devs noted they had no control over auth since auth can only go through Thunderbird itself. So if a plugin needs a specialized auth for some reason they're out of luck, or they have to bake it into their plugin. If instead the auth was modular and a plugin itself, then it's just a matter of what folks install for auth and everything else just uses what's available - or perhaps even requires some selection of plugins.

I assume that Microsoft's OAuth 2.0 implementation within IMAP is different than EWS; most notably the fact that the former hasn't been confirmed to actually work yet.

I don't know all the details. I know Microsoft has multiple methods of authentication and it what users get to use is often based on their organization's configuration and even licenses available. For instance, Microsoft does have an Application Key style auth that works with Basic Auth mechanisms but the org has to have the correct AD/ADFS configuration and then also pay for that as an extra. So the choices are often well out of the hands of the individual mail users - we're just stuck. OAuth is another possibility; ModernAuth is yet another - and even there you can set it up to integrate with a web browser too so the browser handles all the SAML negotiations and you just get a token from the browser. It's a complex web of options.

(By the way, any developer who needs information about obtaining an O365 account for testing purposes please contact me directly)

Awesome!

(In reply to bm_witness from comment #43)

(In reply to Jesse Thompson from comment #42)
When I could I was using the ExchangeCalendar plugin to get Lightning and Exchange working together; however, the devs noted they had no control over auth since auth can only go through Thunderbird itself. So if a plugin needs a specialized auth for some reason they're out of luck, or they have to bake it into their plugin. If instead the auth was modular and a plugin itself, then it's just a matter of what folks install for auth and everything else just uses what's available - or perhaps even requires some selection of plugins.

Does the OWL plugin handle the auth directly or through Thunderbird itself? It seems to function ok. It renders the login page of our SAML identity provider and traverse through multi-factor prompts just fine and it manages to obtain a valid session token through all of that (once I allowed 3rd party cookies for the relevant domains)

I assume that Microsoft's OAuth 2.0 implementation within IMAP is different than EWS; most notably the fact that the former hasn't been confirmed to actually work yet.

I don't know all the details. I know Microsoft has multiple methods of authentication and it what users get to use is often based on their organization's configuration and even licenses available. For instance, Microsoft does have an Application Key style auth that works with Basic Auth mechanisms but the org has to have the correct AD/ADFS configuration and then also pay for that as an extra. So the choices are often well out of the hands of the individual mail users - we're just stuck. OAuth is another possibility; ModernAuth is yet another - and even there you can set it up to integrate with a web browser too so the browser handles all the SAML negotiations and you just get a token from the browser. It's a complex web of options.

Complex indeed. The application password option has been available for a long time but it only works if the org is using Azure AD (cloud) for two-step/multi-factor authentication, and won't work for orgs using Duo or some other multi-factor authentication solution. I can't recall off the top of my head if it's further complicated by the org's chosen authentication provider - Azure AD handling authentication in the cloud VS. federated with ADFS (WS-Trust, WS-Fed) VS. federated via SAML. Needless to say, this is not a viable solution for many orgs who enjoy the local control/flexibility that Microsoft allows. Most importantly, it's not a Modern Auth / OAuth 2.0 flow.

Microsoft's stated commitment to add OAuth 2.0 support to their IMAP implementation should obviate that complexity by being more in line with standards. Assuming the IMAP client recognises the XOAUTH capability, assuming Microsoft's implementation isn't weird, assuming Microsoft actually delivers on their commitment, etc...

Does the OWL plugin handle the auth directly or through Thunderbird itself? It seems to function ok.

Directly. If I understood the developers correctly, they invested a lot of effort to make MFA work.

Having IMAP enabled in Office 365 is causing heavy brute force attacks through this unprotected protocol on user/passwords. We need to disable IMAP (and will be forced to do so by Microsoft in October). Thunderbird is a favorite client for a lot of our users so I hope this will be addressed any time soon.

You are aware that there are add-ons that allow TB to connect to an exchange server and support MFA, right?

(In reply to Jorg K (GMT+1) from comment #47)

You are aware that there are add-ons that allow TB to connect to an exchange server and support MFA, right?

Speaking of add-ons I can see Owl https://addons.thunderbird.net/en-US/thunderbird/addon/owl-for-exchange/ but are there others supporting MFA ?

(In reply to Jorg K (GMT+1) from comment #47)

You are aware that there are add-ons that allow TB to connect to an exchange server and support MFA, right?

No. We've used Calender EWS-plugins with Thunderbird, but as we know there is no such support. Please share if you have better solutions (that not bypass MFA with app passwords) in mind.

We're going to implement this bug in Thunderbird as soon as Microsoft has the oAuth2 support ready for IMAP.

Please share if you have better solutions (that not bypass MFA with app passwords) in mind.

I'm not familiar with these, but last I heard, they do MFA:
https://addons.thunderbird.net/en-GB/thunderbird/addon/owl-for-exchange/
https://addons.thunderbird.net/en-GB/thunderbird/addon/exquilla-exchange-web-services/

(In reply to ulf.lundqvist from comment #46)

We need to disable IMAP (and will be forced to do so by Microsoft in October).

Eh? What are you talking about?

I don't see any news items related to Microsoft removing IMAP access for Office 365 customers, much less within the year. Normally for something this big, they'd give a heads up at least a year or two before doing so.

If you know something, please elaborate, because I use IMAP all over the place here, and if I lost IMAP connectivity (and didn't have another option to use TB with Office 365), I'd be one very, very unhappy camper.

(In reply to Charles from comment #52)

(In reply to ulf.lundqvist from comment #46)

We need to disable IMAP (and will be forced to do so by Microsoft in October).

Eh? What are you talking about?

I don't see any news items related to Microsoft removing IMAP access for Office 365 customers, much less within the year. Normally for something this big, they'd give a heads up at least a year or two before doing so.

If you know something, please elaborate, because I use IMAP all over the place here, and if I lost IMAP connectivity (and didn't have another option to use TB with Office 365), I'd be one very, very unhappy camper.

Sorry, I meant Microsoft will be removing support for IMAP with basic auth in October.

(In reply to ulf.lundqvist from comment #53)

Sorry, I meant Microsoft will be removing support for IMAP with basic auth in October.

Oh, ok, sorry...

(In reply to Magnus Melin [:mkmelin] from comment #50)

We're going to implement this bug in Thunderbird as soon as Microsoft has the oAuth2 support ready for IMAP.

We spoke with someone on the Exchange Online product team. He said that their consumer "Live ID" services (e.g. hotmail) have supported OAuth over IMAP for years, which is why we noticed that their IMAP server advertises the XOAUTH capability (both Office 365 and the Live/consumer email services share the same endpoints).

In theory, Thunderbird could support OAuth today similar to how it works with Gmail (without a 3rd party addon). Is there something nuanced with Microsoft's OAuth capability that is preventing Thunderbird from recognizing it as valid?

(In reply to Jesse Thompson from comment #55)

(In reply to Magnus Melin [:mkmelin] from comment #50)

We're going to implement this bug in Thunderbird as soon as Microsoft has the oAuth2 support ready for IMAP.

We spoke with someone on the Exchange Online product team. He said that their consumer "Live ID" services (e.g. hotmail) have supported OAuth over IMAP for years, which is why we noticed that their IMAP server advertises the XOAUTH capability (both Office 365 and the Live/consumer email services share the same endpoints).

In theory, Thunderbird could support OAuth today similar to how it works with Gmail (without a 3rd party addon). Is there something nuanced with Microsoft's OAuth capability that is preventing Thunderbird from recognizing it as valid?

O365 gets subjected to a lot of customer configuration stuff and how AD/ADFS is integrated; not all features are available in all configurations.

(In reply to Magnus Melin [:mkmelin] from comment #50)

We're going to implement this bug in Thunderbird as soon as Microsoft has the oAuth2 support ready for IMAP.

It'd really be great if this was setup so that users could more easily add additional authentication methods, then someone could contribute a Modern Auth plugin for when OAUTH2 isn't available, or when other methods are provided by a configuration, or other parties (non-Microsoft) have other auth mechanisms. Allowing the community to support plugins to handle this would help with turn-around.

(In reply to Jorg K (GMT+1) from comment #47)

You are aware that there are add-ons that allow TB to connect to an exchange server and support MFA, right?

The problem is that such support has to be implemented in any plugin that wants/needs to use it. There's no cross-support right now. There really needs to be an auth plugin system so that kind of thing can be shared.

O365 will disable basic auth for IMAP in October: https://techcommunity.microsoft.com/t5/exchange-team-blog/basic-auth-and-exchange-online-february-2020-update/bc-p/1200333 The blog post (and the prior one linked from it) seems to suggest that OAuth for IMAP should work. I just tried with Thunderbird 68.5.0 64b after enabling OAuth for our tenant but I still get "The IMAP server outlook.office365.com does not support the selected authentication method". So I'm not sure whether this is supposed to work or not.

To help this get implemented, here is the basic work outline as I understand it..

1. Thunderbird will need to support the PCKE flow for OAUTH. This matches the flows that MSAL uses with working with a Public Client and allows omitting the client_secret from the exchange (unlike Google which supports PCKE but still requires the client_secret)
2. Thunderbird Org will need to obtain a client_id from some Azure tenant. Ideally the client_id would be configured to support consumer (outlook.com) and O365 scenarios. MS has some special stuff required for this
3. The actual OAUTH 2.0 flow is quite similar to Google's, MS documents it here:
   https://docs.microsoft.com/en-us/azure/active-directory/develop/v2-oauth2-auth-code-flow
4. Requesting a long lived refresh token is done with the offline_access scope, which is different from Google
5. MS has not documented yet, but at least for outlook.com the scope is apparently 'wl.imap' which will grant access to SMTP and IMAP.
6. As a 'future goal' thunderbird could use the windows APIs to get the bearer token. This provides single sign and more robust security. No idea how to do this, but AFAIK their open source .NET authentication libraries manage it.

The rest of the flow looks basically the same as Google's OAUTH 2.0 flow, just with different base URLs. Presumably this should work for free outlook.com accounts today, and is probably worth Thunderbird implementing just for that. MS has said IMAP oauth support for O365 'is rolling out now'

Just looking at the existing OAUTH code this looks like a fairly simple extension to where things are now.

Hope this helps, it would fantastic to see this support in Thunderbird as many of us are struggling with IT departments blocking IMAP & Thunderbird. Offering an OAUTH option will make it easier to get it unblocked, as the usual blocking reason is security by banning basic auth.

(In reply to zoltan.lehoczky from comment #57)

O365 will disable basic auth for IMAP in October: https://techcommunity.microsoft.com/t5/exchange-team-blog/basic-auth-and-exchange-online-february-2020-update/bc-p/1200333 The blog post (and the prior one linked from it) seems to suggest that OAuth for IMAP should work. I just tried with Thunderbird 68.5.0 64b after enabling OAuth for our tenant but I still get "The IMAP server outlook.office365.com does not support the selected authentication method". So I'm not sure whether this is supposed to work or not.

OAtuh Support for POP3 & IMAP currently being rolled out.

"OAuth Support for POP3/IMAP4

We are adding OAuth support to POP3 and IMAP4 to improve security of these legacy protocols."

Link: https://www.microsoft.com/en-us/microsoft-365/roadmap?filters=&searchterms=61303

Should start to work again fairly soon i hope

I'm looking forward to test this

Needed to do more clean-up before fixing the actual issue.

With these patches IMAP access is working. (POP3 and SMTP are in the patch, but server side not yet ready.)

For the test accounts I sent you, to do account setup you need to

• clone https://github.com/thundernest/autoconfig
• change the ispdb/outlook.com file to use "OAuth2" authentication for IMAP instead of password-cleartext
• start it. You can put this in a script:

#!/usr/bin/python

import SimpleHTTPServer
import SocketServer

PORT = 8000

Handler = SimpleHTTPServer.SimpleHTTPRequestHandler
Handler.extensions_map.update({
    '.com': 'text/xml',
});

httpd = SocketServer.TCPServer(("", PORT), Handler)

print "Serving at port", PORT
httpd.serve_forever()

• set mailnews.auto_config_url http://localhost:8000/ispdb/

See attachment 9135403 [details] [diff] [review]. For O365, the OAuth2 scopes for SMTP and IMAP are different.

I've just pulled from the daily build and applied your patches. I am able to connect and retrieve my emails perfectly fine with IMAP and 2FA.
However, things don't work yet for SMTP. It does get me as far as asking me for 2FA, but then nothing happens and I get the generic error message below and response from the server.

Does it mean we have to wait for Microsoft to enable this functionality?

Sending of the message failed.
Failed due to unexpected error 80004005. No description is available.
The message could not be sent using Outgoing server (SMTP) smtp.office365.com for an unknown reason. Please verify that your Outgoing server (SMTP) settings are correct and try again.

The response from the server is:

I/SMTP SMTP Response: 454 4.7.120 XOAUTH2 authentication is not enabled [YQXPR0101CA0038.CANPRD01.PROD.OUTLOOK.COM]

Yes, the server side is not ready for SMTP yet.

SMTP. It does get me as far as asking me for 2FA

FWIW, we should not ask for 2FA twice, once for IMAP and once for SMTP. My comment 75 addresses this.
Once you fix that, the code changes get a lot simpler, too.
This would be the same that we do for Google, Yahoo and the others, where we also have only one OAuth2 request for all protocols.

I'll land some preparation parts now and see what/if we can do for the scope sharing.

(In reply to Ben Bucksch (:BenB) from comment #71)

Comment on attachment 9135400 [details] [diff] [review]
bug1528136_nonglobal_oauthsettings.part4.patch

Could you please explain why you need 2 separate OAuth settings for IMAP and
SMTP?

We have different ones for yandex. I agree it's not ideal and I'll make O365 use the same ones. There is one slight drawback, in that since they have different scopes for imap pop and smtp we need to list all three scopes when we ask for them, so the list shown to users lists "read mail permissions" twice.

(In reply to Ben Bucksch (:BenB) from comment #78)

Comment on attachment 9135400 [details] [diff] [review]
bug1528136_nonglobal_oauthsettings.part4.patch

I see no reason to split this. You should get all permissions at the same time.

Having them global like this breaks yandex. It's also wrong for cases where you'd want to, say, configure a different smtp server.

(In reply to Ben Bucksch (:BenB) from comment #74)

Comment on attachment 9135402 [details] [diff] [review]
bug1528136_o365-backout-a5a26b1ee9ec.patch

This part must stay, for the reasons mentioned. It was not only about MFA.

Since going forwards all setups would use OAuth2 this has no place anymore. It was used mainly to detect MFA which is clearly not needed anymore. The other case it affected is when you get a mailbox which has got imap disabled. But we can't really detect that now . Cases like not allowing access (in the oauth dialog) or cancelling the window are both "failures" , and not such that we should cause any selection changes due to them.

IMAP and SMTP support are now working. POP will be enabled soon, but we need bug 1631492 for it to work properly.

Gentlemen, what exactly happened here? Attachment 9135402 [details] [diff] landed without review, so that alone would be a reason to back it out.

Also, we've been discussing this at length at end of 2019 and the beginning of 2020 with Alex as mediator: The solution in bug 1592258 was to test IMAP, and if it wasn't working, not to offer it. So why can that be removed? Sure, O365 may offer MFA support now which will make more scenarios work, but not all.

That backout patch was necessary in this bug to make the bug1528136_o365_oauth2_support.patch work as it introduces built in handling of O365 oauth, which is the patch I reviewed and approved.

I'm not sure why Magnus landed the backout patch without this, probably needs some changes before landing and meanwhile he landed the other patches in preparation for this to make it easier to implement it?
I don't know, and of course I don't want to speak for him, but I assume the objective is to integrate those features into TB core.

Aleca wrote:

That backout patch was necessary ... to make the ... oauth2_support.patch work

The backout wasn't necessary, no. It would mean that the OAuth2 login happens earlier, but that's fine. We already did the password login at this point. We might just need small adaptions.

Contrary to what Magnus said, my backed out patch did indeed detect the case that IMAP was disabled by the domain admin, and it also detected other configurations.

These users are now broken by the backout, with a non-working default config.

Given that this is a very complicated matter, which has been discussed for months in the TB Council and in the bug, and all the reasons are in bug 1592258, I am suggesting that the discussion continues in that bug, because that patch belongs there.

(In reply to Ben Bucksch (:BenB) from comment #93)

Given that this is a very complicated matter, which has been discussed for months in the TB Council and in the bug, and all the reasons are in bug 1592258, I am suggesting that the discussion continues in that bug, because that patch belongs there.

That would be nice except that when normal mortals log in to see Bug 1592258, they are told that "You are not authorized to access 1592258."

I appear to be seeing more and more of the not authorized or work done on bugs I have been following for years done in new bugs. What is going on folks? Is this bug about 2FA which I would not like to see? Applications passwords are bad enough. Or oAuth?

@Matt: This bug is about implementing OAuth for Microsoft 365 (formerly Office365) email. Thunderbird doesn't have any choice in the matter, because Microsoft was planning to disable the normal password-based auth for IMAP and SMTP in October, so TB is forced to implement that. With the OAuth2 support, TB will also automatically support MFA.
(FWIW, I am not objecting to any of this. I objected to how it happened.)

@BenB - Matt wasn't commenting on this bug, Matt was commenting on Bug 1592258 which is locked down and we can't view for some reason. Therefore, the suggestion to continue the discussion in Bug 1592258 won't work until someone unlocks that bug.

And in general, locked bugs are antithetical to work in Open Source. They may be necessary for short periods if there are embargoed security issues, but they shouldn't be used for routine work.

Oh, I see. Sorry, I misunderstood. FWIW, it wasn't me who locked this bug. But believe me, you don't even want to read the 167 comments there. Frankly, I want my life time back that I spent in that bug!
The long and short of it is that Office365 is extremely configurable by admins in many different ways, and there is no way for us to know whether a certain email account will work with IMAP or IMAP + OAuth2 MFA or only with Exchange protocols, so before we're suggesting IMAP as configuration, we're testing whether it actually works. The result is the patch which was backed out in https://hg.mozilla.org/comm-central/rev/62453d8cdbb7 That's why I was so upset that this was backed out. The bug here should fix the MFA + IMAP case, but none of the others, so now we're suggesting a non-working config for users who cannot use IMAP.

Let's be constructive.

• The backed out patch is still needed for some users.
• The backed out patch can inherently work with the fix here.
  • It uses VerifyLogin(), which is the same function that we use at the end of the account creation. The only difference is that we'd do the auth earlier. But there is nothing in the code that you backed out that is inherently incompatible with the fix here.
  • If there is a problem, I'd be happy to help fix it. Please describe the problem in detail, so that I can fix it. (As long as it's not just artificially constructed.)

Most importantly:

• The fix here in this bug does not work for me. As mentioned above, I was not able to log in with any of the Office365 test accounts that I have. (Apart from those that have IMAP enabled and MFA disabled, i.e. those accounts that worked even before this fix.) Just to ensure that the problem is not some sort of setting in my test accounts, I just created another completely new Office365 test domain and new user. I've created a new company account and a new domain, used Microsoft 365 Business Basic, changed none of the settings, and only enabled MFA for that test user. As admin, I activated MFA for that user, and as the user, I set up an TOTP app as 2 factor auth. IMAP is activated for that user. I changed nothing else, completely virgin account with MFA and IMAP. I tried to log in with Thunderbird trunk - with the fix here -, went through the account creation, IMAP is selected, and when I click [Done], I get: "Unable to log in at server. Probably wrong configuration, username or password".

So, from what I see, the fix here does not work, with a standard Microsoft365 account with IMAP and MFA. Could you please tell why you think this fix it correct?

I confirmed that Thunderbird Daily is able to successfully authenticate using OAuth2 via IMAP and SMTP against our O365 tenant. SMTP did not require a separate prompt, and is definitely using OAuth2 against smtp.office365.com (I have basic authentication disabled on my account, regardless of protocol).

Thanks everyone! This is going to make about 2500 of our users happy (aside from the chore of needing to reconfigure their clients, of course).

I have been watching this bugreport as well, thank you for all the efforts!
I tried the Thunderbird Daily (77.0a1), but I have some trouble signing in because our IdP requires the support of WebSocket-support in order to perform our MFA-login.

Is there an option to enable WebSocket-support for the embedded that is used for the OAuth2 token retrieval and prior authentication process?
Should I file a new bug report for this?

I tried this on my two office365 accounts that have modern authentication. It worked on one and did not on the other.

In both cases it did not autodetect the need for OAUTH2, so there is probably a usability issue lurking there.

For the account that didn't work (redcross.org, which uses an external 2FA provider): setting the account to OAUTH2 generated all the 2FA prompts in a new window as expected. I tried turning on the log (MOZ_LOG=IMAP:5,timestamp). I didn't see much that was interesting, it just said "authenticate failed"

2020-04-29 00:36:27.208000 UTC - [(null) 24544: IMAP]: I/IMAP 23F12000:outlook.office365.com:NA:ProcessCurrentURL: entering
2020-04-29 00:36:27.208000 UTC - [(null) 24544: IMAP]: I/IMAP 23F12000:outlook.office365.com:NA:ProcessCurrentURL:imap://xxxxxxxxxxxx%40redcross%2Eorg@outlook.office365.com:993/select%3E%5EINBOX:  = currentUrl
2020-04-29 00:36:27.325000 UTC - [(null) 24544: IMAP]: D/IMAP ReadNextLine [stream=2C782D00 nb=160 needmore=0]
2020-04-29 00:36:27.325000 UTC - [(null) 24544: IMAP]: I/IMAP 23F12000:outlook.office365.com:NA:CreateNewLineFromSocket: * OK The Microsoft Exchange IMAP4 service is ready. [xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx]^M
2020-04-29 00:36:27.328000 UTC - [(null) 24544: IMAP]: I/IMAP 23F12000:outlook.office365.com:NA:SendData: 1 capability^M
2020-04-29 00:36:27.343000 UTC - [(null) 24544: IMAP]: D/IMAP ReadNextLine [stream=2C782D00 nb=115 needmore=0]
2020-04-29 00:36:27.343000 UTC - [(null) 24544: IMAP]: I/IMAP 23F12000:outlook.office365.com:NA:CreateNewLineFromSocket: * CAPABILITY IMAP4 IMAP4rev1 AUTH=PLAIN AUTH=XOAUTH2 SASL-IR UIDPLUS ID UNSELECT CHILDREN IDLE NAMESPACE LITERAL+^M
2020-04-29 00:36:27.346000 UTC - [(null) 24544: IMAP]: D/IMAP ReadNextLine [stream=2C782D00 nb=28 needmore=0]
2020-04-29 00:36:27.346000 UTC - [(null) 24544: IMAP]: I/IMAP 23F12000:outlook.office365.com:NA:CreateNewLineFromSocket: 1 OK CAPABILITY completed.^M
2020-04-29 00:36:27.346000 UTC - [(null) 24544: IMAP]: D/IMAP Try to log in
2020-04-29 00:36:27.346000 UTC - [(null) 24544: IMAP]: D/IMAP IMAP auth: server caps 0x800087635, pref 0x800000000, failed 0x0, avail caps 0x800000000
2020-04-29 00:36:27.346000 UTC - [(null) 24544: IMAP]: D/IMAP (GSSAPI = 0x1000000, CRAM = 0x20000, NTLM = 0x100000, MSN = 0x200000, PLAIN = 0x1000, LOGIN = 0x2, old-style IMAP login = 0x4, auth external IMAP login = 0x20000000, OAUTH2 = 0x800000000)
2020-04-29 00:36:27.346000 UTC - [(null) 24544: IMAP]: D/IMAP Trying auth method 0x800000000
2020-04-29 00:36:27.346000 UTC - [(null) 24544: IMAP]: D/IMAP IMAP: trying auth method 0x800000000
2020-04-29 00:36:27.346000 UTC - [(null) 24544: IMAP]: D/IMAP XOAUTH2 auth
2020-04-29 00:36:27.675000 UTC - [(null) 24544: IMAP]: I/IMAP 23F12000:outlook.office365.com:NA:SendData: Logging suppressed for this command (it probably contained authentication information)
2020-04-29 00:36:28.274000 UTC - [(null) 24544: IMAP]: D/IMAP ReadNextLine [stream=2C782D00 nb=27 needmore=0]
2020-04-29 00:36:28.274000 UTC - [(null) 24544: IMAP]: I/IMAP 23F12000:outlook.office365.com:NA:CreateNewLineFromSocket: 2 NO AUTHENTICATE failed.^M
2020-04-29 00:36:28.274000 UTC - [(null) 24544: IMAP]: D/IMAP authlogin failed
2020-04-29 00:36:28.274000 UTC - [(null) 24544: IMAP]: D/IMAP Marking auth method 0x800000000 failed
2020-04-29 00:36:28.274000 UTC - [(null) 24544: IMAP]: D/IMAP IMAP auth: server caps 0x800087635, pref 0x800000000, failed 0x800000000, avail caps 0x0
2020-04-29 00:36:28.274000 UTC - [(null) 24544: IMAP]: D/IMAP (GSSAPI = 0x1000000, CRAM = 0x20000, NTLM = 0x100000, MSN = 0x200000, PLAIN = 0x1000, LOGIN = 0x2, old-style IMAP login = 0x4, auth external IMAP login = 0x20000000, OAUTH2 = 0x800000000)
2020-04-29 00:36:28.274000 UTC - [(null) 24544: IMAP]: D/IMAP No remaining auth method
2020-04-29 00:36:28.284000 UTC - [(null) 24544: IMAP]: E/IMAP login failed entirely
2020-04-29 00:36:28.317000 UTC - [(null) 24544: IMAP]: I/IMAP 23F12000:outlook.office365.com:NA:ProcessCurrentURL: aborting queued urls
2020-04-29 00:36:28.320000 UTC - [(null) 24544: IMAP]: I/IMAP 23F12000:outlook.office365.com:NA:TellThreadToDie: close socket connection
2020-04-29 00:36:28.320000 UTC - [(null) 24544: IMAP]: D/IMAP ImapThreadMainLoop leaving [this=23F12000]

Sorry, should have said: error was generated using Thunderbird Daily 2020-04-28, 32 bit, on Windows 10

Hello all, i am following this bug for about two months now after my company changed the settings on our office365 accounts to use only modern authentication.

I can verify what Alessandro said, i can create an IMAP account only if i first login to outlook with firefox (login with chrome or other browser will not work). Only then i get a pop up window for MFA asking for my password and verification.
For this to work i must manually choose OAuth2 for authentication in the manual configuration.
This was tested with windows 7 SP1 and Ubuntu 18.04 using Thunderbird Daily 2020-04-28, 32 bit.
Hope this helps somehow!

Will there be a fix for POP3 also?

That explains why it didn't work for me. I did log in to the account with Firefox, but with a different IP address. I also am using the account creation dialog, obviously.

It seems there are 3 independent (!) problems:

1. It doesn't automatically detect the authentication method
   -> The auth method needs to be added to the config object. We cannot do that in the ISPDB, because TB 68 would then also try to do it for Offic365 and fail, so the only idea I have right now is to make a special hack for Office365 in the code, until TB 68 is dead. I might have a better idea later.
2. The OAuth2 doesn't actually grant access, unless a browser login happened before.
   -> No idea what the fix would be. We probably need to emulate a Firefox browser more. I'd start with the user agent string. But only for this domain, obviously.
3. Users with disabled IMAP and other custom settings get a non-working config.
   -> The backed out patch fixes that.

(In reply to Ben Bucksch (:BenB) from comment #108)

2. The OAuth2 doesn't actually grant access, unless a browser login happened before.
   -> No idea what the fix would be. We probably need to emulate a Firefox browser more. I'd start with the user agent string. But only for this domain, obviously.

I have slightly different experience: I have tried Thunderbird Daily build 77.0a1 (2020-04-28) on Mac OS X, and it did work for me without the need to login via Firefox (I don't have Firefox installed at all). My default browser is set to Chrome, and I was logged into Microsoft account on it, but it was not the Firefox. I will try to find some time to do the "clean" test (log out of my Outlook account in browser, remove account from Thunderbird, and document steps to set up the authentication. It didn't work straight away, and it did take few iterations to make it work, but the bottom line is it did work in the end with an account with MFA enabled and enforced.

The OAuth2 doesn't actually grant access, unless a browser login happened before.
-> No idea what the fix would be. We probably need to emulate a Firefox browser more. I'd start with the user agent string. But only for this domain, obviously.

Instead of emulating Firefox Browser, is there a way to integrate with the default browser on the host OS? I don't think Thunderbird should enforce operation with Firefox as being required, and emulating Firefox would probably not be a good idea either.

That said - enabling the use of an extension (such as https://addons.mozilla.org/en-US/firefox/addon/access-panel-extension/) would probably be a good thing.

Q: Is there any reason why Thunderbird cannot have an extension API for authentication? I called that out above (Comments 3, 56) but that seems to have gotten ignored in the rest of this thread.

https://bugzilla.mozilla.org/show_bug.cgi?id=1528136#c3
https://bugzilla.mozilla.org/show_bug.cgi?id=1528136#c56

Q: is this truly resolved? I still see people reporting here that it's not fully working, and further no one has answered my question from Comment #110 either.

is there a way to integrate with the default browser on the host OS?

No. The only OS API for browsers is to launch a URL, it's "fire and forget", but we need to control and observe the page flow, to know when the login finished, and we even need to get information from the page to continue the OAuth2 flow. Otherwise the whole process does not work.

(In reply to Ben Bucksch (:BenB) from comment #112)

is there a way to integrate with the default browser on the host OS?

No. The only OS API for browsers is to launch a URL, it's "fire and forget", but we need to control and observe the page flow, to know when the login finished, and we even need to get information from the page to continue the OAuth2 flow. Otherwise the whole process does not work.

I was more referring to my other question - I'm sure there are ways to integrate with different browsers - outside of OS APIs - that can get around that as I've seen some applications do it in order to get cookies stored by the browser to join SSO sessions. I don't know how easy/hard that is to do.

The question from Comment #110 that I'm primarily interested in having a response to is:

Q: Is there any reason why Thunderbird cannot have an extension API for authentication? I called that out above (Comments #3, Comment #56) but that seems to have gotten ignored in the rest of this thread.

(In reply to Ben Bucksch (:BenB) from comment #112)

is there a way to integrate with the default browser on the host OS?

No. The only OS API for browsers is to launch a URL, it's "fire and forget", but we need to control and observe the page flow, to know when the login finished, and we even need to get information from the page to continue the OAuth2 flow. Otherwise the whole process does not work.

RFC 8252 recommends using the system browser, and against using internal browsers.

This is what most new apps are doing these days. The redirect URL can be something that triggers an external program to pass the first response back. eg VS Code on Linux triggers an xdg-open for the response. You can also use a localhost http url for the redirection.

The system browser is much better. For instance Chrome has a plugin that links to windows integrated authentication and can use the system TPM to authorize OAUTH tokens.

We're getting close to 78 beta.
Should we open a follow up bug to fix all the remaining issues with MFA?

As i understand this is not working so why it is marked resolved?
I searched all bugs related but there is no solution about thunderbird and Office 365 MFA.
If this doesn't get resolved many linux users will have to find other solution for connecting to their account. At least windows users can switch back to Outlook!

In general, we have 4 different user types/configurations on Office365:
User types:

1. IMAP enabled, MFA disabled
2. IMAP enabled, MFA enabled
3. IMAP disabled, with or without MFA
4. Other custom configuration through Conditional Access policies or other uncommon settings

Status of each:
User types:

1. Was working before and is still working
2. Is supposed to be fixed in this bug, but not working yet
3. Needs Exchange protocols, and should default to them. This was working before and broke with the backout.
4. Like 3.

From my understanding, here are the remaining problems. They need several fixes:
Bugs:

1. The code problems mentioned in the code reviews above.
2. The fix will not work at all (in the account creation) until the ISPDB entry for Office365 is changed to OAuth2. This needs to wait until TB 68 is phased out. Alternatively, the code in TB 78 could be hacked to use OAuth2 for Office365. This would depend on the backed out patch.
3. The backed out patch needs to go back in. It uses the exact same function verifyConfig() as is run after clicking [Done], so if this fails, then it's not due to the patch, but it's because the first OAuth2 login fails or something else is buggy in the OAuth2 code. See next point.
4. It appears that the OAuth2 process fails the first time.
5. Several people reported that they have to log in via the browser first before it works in Thunderbird. Aleca said that he explicitly checked [x] Trust this device, which made the difference.

Re bug 4 in my last comment:

Reproduction:
Steps:

1. Code: TB trunk, plus the backed out patch re-applied, plus a fix for point 2 above.
2. Create a new TB profile and start the account creation dialog
3. Enter email address with IMAP and MFA (type 2 above), enter password
4. Webpage popup with Office365 login. Enter password and MFA code, and confirm access for TB
5. The window closes, but the login fails (that's the bug 4).
6. Consequently, (o) Exchange is selected as default protocol. (That's a consequence of the OAuth2 process failing, not the fault of the backed out patch.)
7. Manually change back to (o) IMAP
8. Click [Done]
9. OAuth2 works now, without webpage, because the user is already logged in.
10. The account is set up correctly.
    So far, it looks like the backed out patch is at fault. But now it gets interesting:
11. Delete the profile and create a completely new TB profile.
12. Enter email address with IMAP and MFA (type 2 above), enter password
13. Webpage popup with Office365 login. Enter password and MFA code, and confirm access for TB.
14. The window closes, but the login succeeds (compare with step 5).
15. Consequently, (o) IMAP is selected as default protocol.
16. Click [Done]
17. OAuth2 works now, without webpage, because the user is already logged in.

So, we do the exact same thing twice, in new profiles each, and the first time it fails and the second time it works.
Now, wait 1 hour or so (I haven't measured). Then, repeat steps 1.-17. You will see that it fails again the first time and it work the second time. I can reproduce this fairly consistently, with several different Office365 accounts on different domains.

Here's a possible explanation. It's a theory at this point, but an educated guess based on our development experience with Office365 login:
Tokens:

1. OAuth2 uses access token and refresh token.
   1.1. This (or Token 3 below) explains the difference between Steps 4 and 9. We call verifyConfig() in both cases, but the second login already is authenticated, so the second login uses a different code path where we're already logged in, and that works.
2. Additionally, there's an auth code (not OAuth2 access token) in the URL at the end of the webpage sequence. We might need to get that out and pass it to OAuth2 server. None of this is standardized, but it was needed in our case.
   2.1. This might fix the failure in Step 5 == Bug 4.
   2.2. This might also fix Bug 5.
3. Additionally, Microsoft uses cookies to store the login.
4. Additionally, Microsoft appears to remember the combination of IP address and time. If the same request comes in again from a known IP address within a given time, even without any cookies, it gets different results than a completely cold request.
   4.1. This explains the difference between Steps 5. and 14.
   4.2. Obviously, we cannot rely on that, so see Token 2.1.

Thank you Ben for the thorough explanation and detailed analysis.
Indeed, upon testing the different configurations with the help of Ben, these 4 scenarios need to be properly handled to offer users a working configuration and a easy path.
Right now sometimes it works sometimes it doesn't.

Magnus, we should reopen this bug and nail down the issue with MFA on IMAP, and for the other cases which Thunderbird can't handle, we should put back the backed out patch.

A couple of extra pointers:

The fix will not work at all (in the account creation) until the ISPDB entry for Office365 is changed to OAuth2. This needs to wait until TB 68 is phased out.

Ben also suggested adding the line this._currentConfig.incoming.auth = Ci.nsMsgAuthMethod.OAuth2; in the backed out patch, which force the use of OAuth2 instead of password-cleartext, to allow 78 to temporarily work with this fix without the need to edit the ISPDB entry.

It appears that the OAuth2 process fails the first time.

We can investigate and explore a proper solution in a follow up bug to be sure we're returning the correct configuration.
It seems that Microsoft caches some device info the first time you attempt to log in. So the first attempt fails but it actually generates the OAuth token, and if we try a second time with the same credentials, the log in is successful.

the first attempt fails

OAuth2 doesn't actually specify how to hand over from the web page login to the OAuth2 server REST API, and this is custom stuff in Microsoft, so it's not surprising me that it might work for Google login and not for Microsoft login, without a custom adaption. I suspect that there's something wrong in our OAuth2 implementation for Office365. I think it is part of this bug here. The above description should give enough pointers where to look. We went through the same pains, and many more. It took us months to develop and investigate this and many other cases.

Is this solved? I just installed latest release and can't login.

FWIW, tested nightly 2020-05-22 Linux, with two tenants (using Azure MFA and Duo) with success.

The setup flow is a bit wonky though, it tends to hide the OAUTH2 option in the combo box for some reason.

(In reply to Jason Gunthorpe from comment #123)

FWIW, tested nightly 2020-05-22 Linux, with two tenants (using Azure MFA and Duo) with success.

The setup flow is a bit wonky though, it tends to hide the OAUTH2 option in the combo box for some reason.

Thank you. When will this be released in stable release?

Has anyone been able to get this work with outlook.office365.com yet? I downloaded the 2020-05-28 nightly and have authentication set to OAuth2, I've tried StartTLS and SSL for the security, but it fails all login attempts. For now I can still log in with plain authentication (with the same credentials) but that won't last much longer. It seems kind of odd that this bug was closed when it appears that myself and others are still have issues accessing the very site that generated this bug report.

Jeff, yes it works. Here are some instructions: https://kb.wisc.edu/102005

You may want to confirm with your administrator (if you aren't one) that you don't have the IMAP, POP, and SMTP protocols disabled on your mailbox. Many Exchange administrators have traditionally used that as a tactic to restrict access for HIPAA compliance or otherwise just forcing users into using a common client (Outlook) to reduce their IT support costs.

Jeff. For what it’s worth Microsoft delayed the depreciation of Basic Auth to 2nd half of 2021, firm date still pending.
https://techcommunity.microsoft.com/t5/exchange-team-blog/basic-authentication-and-exchange-online-april-2020-update/ba-p/1275508

I did get it to work on nightly build although it took some fiddling with the settings even after the Oauth prompt came and went it still didn’t immediately connect. Relaunching the app and more fiddling it finally connected. This was just after it was fixed. It sure if it’s improved since then. I suppose your experience could be similar to mine. I have no guidance other than “fiddle away til it works”. :)

I too would like to know when this will be released in ESR or current release. Does anyone know usually how long after a big fix is completed, does it get merged with current?

Thanks for the responses, folks!

@Jesse -- It appears the instructions you linked simply took a regular IMAP account and changed the authentication to OAuth2. Oddly enough that's exactly what I did before, but I went ahead and created a new account and made the change. As with before, it popped up a dialog asking me to log in to outlook.com. Previously when I closed and restarted thunderbird it failed to re-authenticate, however this time it jumped right back in to downloading my mailbox. I'll keep an eye on it, but for the moment it appears to be working this time.

@emailthereeves -- Unfortunately my employer has set a hard cutoff date of July 1st for us after only first mentioning it in late April. Not only does half our department prefer thunderbird, but we also have one person who requires a screen reader and only uses linux from the command line, so I've been trying to figure out if there's even a solution to getting alpine to work with outlook.com.

So my only concern now is waiting to see if my credentials are renewed after they expire (comments above suggest this happens at least daily). If it continues to work for the rest of the week for me, then I can start helping other department members make the conversion to the nightly build until we have an ESR available.

Just reporting back that it is still working today. However I've been thinking about what I did the first time around, and something occurred to me. I did not originally check the box on the microsoft login screen to remember my credentials and later when it began failing the connection TB never brought up that dialog to ask me to log in again. I'm wondering if the two are related, and if so would this be a bug that needs to be addressed?

(In reply to Jeff Taylor from comment #129)

Just reporting back that it is still working today. However I've been thinking about what I did the first time around, and something occurred to me. I did not originally check the box on the microsoft login screen to remember my credentials and later when it began failing the connection TB never brought up that dialog to ask me to log in again. I'm wondering if the two are related, and if so would this be a bug that needs to be addressed?

I'm not intimately familiar with Azure AD's cloud authentication behavior (we federate to a SAML provider) but it's possible that neglecting to check that box resulted in a very short OAuth session length. I can't say for sure if it should be a bug to fix, but I think that it makes sense that the client should detect and gracefully handle requests for reauthentication.

Per req in comment 117.

I've been testing this again. I no longer see the problem with the error on first login - likely some server bug was fixed.
But reproduction steps were no 100% clear (and difficult), so if people could report how Thunderbird 78 beta works for them that would be appreciated. You still have to manually set oauth2 in the account setup - bug 1648779 will take care of that, and we should also update ISPDB soon.

(In reply to Magnus Melin [:mkmelin] from comment #135)

I've been testing this again. I no longer see the problem with the error on first login - likely some server bug was fixed.
But reproduction steps were no 100% clear (and difficult), so if people could report how Thunderbird 78 beta works for them that would be appreciated. You still have to manually set oauth2 in the account setup - bug 1648779 will take care of that, and we should also update ISPDB soon.

How long does it take for bug fixes to merge into Current Release? Speaking specifically about OAuth for Office365 of course :)

I no longer see the problem with the error on first login

It depends on your account, it's a server-side per-account setting. Did you try with a completely new account? Or with an old account?

I tested with both new, older and a day old account.

(In reply to Jeremy Reeves from comment #136)

How long does it take for bug fixes to merge into Current Release? Speaking specifically about OAuth for Office365 of course :)

For this one it's taken a bit longer since there have been some issues where it's unclear where the problem lays.
Thunderbird 78.0 will be out for manual download in 10 days. It will take a bit longer before we auto-upgrade everybody.

BTW, someone reported earlier it didn't work due to missing websocket support (needed because of some special setup they had). This has now also been fixed.

I just retested on 78.0b3. I still can't log into an office365 account (redcross.org, uses a 3rd party 2 factor auth).

Enter account info, change to "manual", set to oauth2, "advanced config". Got 2 factor auth popup. After that: get "Authentication failure when connecting to server outlook.office365.com".

Login continues to fail after subsequent login attempts.

Deleting and recreating account: don't get 2 factor popup, just login failure (are credentials cached across deleted accounts? That doesn't seem reasonable)

Exiting and restarting thunderbird: no effect.

Neil. That’s likely a result of browser cache.

Deleting an account doesn't currently remove the oauth details - to test it you'd have to go into the Options | Privacy & Security | Passwords. Then delete the oauth details for outlook.office365.com

(In reply to Magnus Melin [:mkmelin] from comment #146)

Deleting an account doesn't currently remove the oauth details - to test it you'd have to go into the Options | Privacy & Security | Passwords. Then delete the oauth details for outlook.office365.com

Thanks; sorry for my ignorance about this.

Deleting the oauth "password" entry causes a new two factor verification (as expected), but after the verification window closes thunderbird still says "Authentication failure when connecting to server outlook.office365.com". So something is getting lost in the handoff back to TB.

I ran tb beta with MOZ_LOG set to IMAP:5. I didn't see anything useful, but perhaps someone else will. Let me know if there are other debugging options that would be useful.

2020-06-27 21:25:16.038000 UTC - [(null) 1240: IMAP]: D/IMAP ImapThreadMainLoop entering [this=2A561800]
2020-06-27 21:25:16.105000 UTC - [(null) 1240: Main Thread]: I/IMAP 2A561800:outlook.office365.com:NA:SetupWithUrlCallback: clearing IMAP_CONNECTION_IS_OPEN
2020-06-27 21:25:16.105000 UTC - [(null) 1240: IMAP]: I/IMAP 2A561800:outlook.office365.com:NA:ProcessCurrentURL: entering
2020-06-27 21:25:16.105000 UTC - [(null) 1240: IMAP]: I/IMAP 2A561800:outlook.office365.com:NA:ProcessCurrentURL:imap://XXXXXXXXXX%40redcross%2Eorg@outlook.office365.com:993/select%3E%5EINBOX:  = currentUrl
2020-06-27 21:25:16.227000 UTC - [(null) 1240: IMAP]: D/IMAP ReadNextLine [stream=33E0B100 nb=160 needmore=0]
2020-06-27 21:25:16.227000 UTC - [(null) 1240: IMAP]: I/IMAP 2A561800:outlook.office365.com:NA:CreateNewLineFromSocket: * OK The Microsoft Exchange IMAP4 service is ready. [QgBZAEEAUABSADAANQBDAEEAMAAwADMAOAAuAG4AYQBtAHAAcgBkADAANQAuAHAAcgBvAGQALgBvAHUAdABsAG8AbwBrAC4AYwBvAG0A]
2020-06-27 21:25:16.230000 UTC - [(null) 1240: IMAP]: I/IMAP 2A561800:outlook.office365.com:NA:SendData: 1 capability
2020-06-27 21:25:16.253000 UTC - [(null) 1240: IMAP]: D/IMAP ReadNextLine [stream=33E0B100 nb=120 needmore=0]
2020-06-27 21:25:16.253000 UTC - [(null) 1240: IMAP]: I/IMAP 2A561800:outlook.office365.com:NA:CreateNewLineFromSocket: * CAPABILITY IMAP4 IMAP4rev1 AUTH=PLAIN AUTH=XOAUTH2 SASL-IR UIDPLUS MOVE ID UNSELECT CHILDREN IDLE NAMESPACE LITERAL+
2020-06-27 21:25:16.262000 UTC - [(null) 1240: IMAP]: D/IMAP ReadNextLine [stream=33E0B100 nb=28 needmore=0]
2020-06-27 21:25:16.262000 UTC - [(null) 1240: IMAP]: I/IMAP 2A561800:outlook.office365.com:NA:CreateNewLineFromSocket: 1 OK CAPABILITY completed.
2020-06-27 21:25:16.262000 UTC - [(null) 1240: IMAP]: D/IMAP Try to log in
2020-06-27 21:25:16.262000 UTC - [(null) 1240: IMAP]: D/IMAP IMAP auth: server caps 0x840087635, pref 0x800000000, failed 0x0, avail caps 0x800000000
2020-06-27 21:25:16.262000 UTC - [(null) 1240: IMAP]: D/IMAP (GSSAPI = 0x1000000, CRAM = 0x20000, NTLM = 0x100000, MSN = 0x200000, PLAIN = 0x1000, LOGIN = 0x2, old-style IMAP login = 0x4, auth external IMAP login = 0x20000000, OAUTH2 = 0x800000000)
2020-06-27 21:25:16.262000 UTC - [(null) 1240: IMAP]: D/IMAP Trying auth method 0x800000000
2020-06-27 21:25:16.287000 UTC - [(null) 1240: IMAP]: D/IMAP IMAP: trying auth method 0x800000000
2020-06-27 21:25:16.287000 UTC - [(null) 1240: IMAP]: D/IMAP XOAUTH2 auth
2020-06-27 21:25:33.561000 UTC - [(null) 1240: IMAP]: I/IMAP 2A561800:outlook.office365.com:NA:SendData: Logging suppressed for this command (it probably contained authentication information)
2020-06-27 21:25:33.809000 UTC - [(null) 1240: IMAP]: D/IMAP ReadNextLine [stream=33E0B100 nb=27 needmore=0]
2020-06-27 21:25:33.809000 UTC - [(null) 1240: IMAP]: I/IMAP 2A561800:outlook.office365.com:NA:CreateNewLineFromSocket: 2 NO AUTHENTICATE failed.
2020-06-27 21:25:33.809000 UTC - [(null) 1240: IMAP]: D/IMAP authlogin failed
2020-06-27 21:25:33.809000 UTC - [(null) 1240: IMAP]: D/IMAP Marking auth method 0x800000000 failed
2020-06-27 21:25:33.809000 UTC - [(null) 1240: IMAP]: D/IMAP IMAP auth: server caps 0x840087635, pref 0x800000000, failed 0x800000000, avail caps 0x0
2020-06-27 21:25:33.809000 UTC - [(null) 1240: IMAP]: D/IMAP (GSSAPI = 0x1000000, CRAM = 0x20000, NTLM = 0x100000, MSN = 0x200000, PLAIN = 0x1000, LOGIN = 0x2, old-style IMAP login = 0x4, auth external IMAP login = 0x20000000, OAUTH2 = 0x800000000)
2020-06-27 21:25:33.809000 UTC - [(null) 1240: IMAP]: D/IMAP No remaining auth method
2020-06-27 21:25:33.816000 UTC - [(null) 1240: IMAP]: E/IMAP login failed entirely
2020-06-27 21:25:33.838000 UTC - [(null) 1240: IMAP]: I/IMAP 2A561800:outlook.office365.com:NA:ProcessCurrentURL: aborting queued urls
2020-06-27 21:25:33.847000 UTC - [(null) 1240: IMAP]: I/IMAP 2A561800:outlook.office365.com:NA:TellThreadToDie: close socket connection
2020-06-27 21:25:33.847000 UTC - [(null) 1240: IMAP]: D/IMAP ImapThreadMainLoop leaving [this=2A561800]

[the log]

The log just says that you had a pref set to force OAuth2, and the OAuth2 login attempt failed.

after the verification window closes thunderbird still says "Authentication failure when connecting to server outlook.office365.com". So something is getting lost in the handoff back to TB.

Yes, it seems that the MS-variant of the OAuth2 protocol is not implemented correctly at the end. See comment 119.

I can confirm the websocket works now using v78beta3.
I was able to open the mailbox and to send and receive messages.

After adding the account to TB, I got that pop-up message "Authentication failure while connecting to server outlook.office365.com"
Looks like a false positive in my case...

Thunderbird 78.0b4:
https://hg.mozilla.org/releases/comm-beta/rev/95c7a87cd30c

I seem to also be having a problem with authentication. My home computer is Debian linux with TB79.0a1 (2020-06-17) and logging in to my work account has been flawless. However I had to go in to the office today, and my work machine (also the same version of Debian and TB) is failing authentication to the same mailbox. I updated it to the 2020-06-29 release and had the same results -- shortly after starting the program I get a notification that authentication with outlook.office365.com has failed. There is no dialog asking me to re-enter my password, nor does the graphical Microsoft dialog appear to ask me to log in to their website again.

I switched my account back to 'normal password' and it connected immediately, so I know TB at least has the correct password stored for me locally. Switching back to OAuth2 once again brought up the authentication error.

Does anyone have suggestions for me to try next time I go back to the office? I only have the one mailbox to work with so I cant create new accounts for testing like others here have done. Thanks!

In reading the comments above I'm a little unclear on whether this is expected to be working in Beta/Nightly builds.

In my testing it appears not to be working yet. What I did:

• Download Daily (79.0a1 (2020-06-29) (64-bit))
• Configure a new account
• Go to manual account configuration to force OAuth as the authentication mechanism
• Set SSL/TLS as the connection security
• Go to the Message view
• Click "Get Messages"
• I get a window with 2FA prompt; I provide the needed credentials
• The 2FA window goes away and my 2FA client reports that it has provided the auth needed
• Click "Get Messages"

I hear a "beep" and see an alert from Thunderbird Daily that tells me of an "Authentication error while connecting to server .xyz.com" (where ".xyz.com" is the domain from my email address and not the "outlook.office365.com" address specified in the account configuration).

Since I saw 78.0b4 mentioned above, I just downloaded and installed it on my home computer, and started it up with --ProfileManager to have it use my existing profile. It presented the graphical Microsoft login screen where I entered my email address and password, clicked the option to save my password, and then got the previously-mentioned authentication error message (and to correct my previous post, it shows the address of my company's exchange server, NOT outlook.office365.com as I stated before -- Thanks Joe McCabe for getting me to double-check this!). As with my work computer, I am never asked to try logging in to Microsoft again.

I shut down 78.0b4 and started 79.0a1 back up again (the version that has been working for me), and was once again given the error that my exchange server failed my authentication. Now I am locked out of my work email completely under OAuth2, and my only option is to use 'normal password' authentication which is going to be shut off after tomorrow. Hope someone can provide a quick answer to get this working again!

beta 4 is not built yet.

@Jeff: until the bug is sorted, you can use DavMail until this is sorted. It's far from perfect but at it more or less works ok.

(In reply to christophe.dubach from comment #157)

@Jeff: until the bug is sorted, you can use DavMail until this is sorted. It's far from perfect but at it more or less works ok.

Are there instructions that I've overlooked on how to make DavMail sit in front of Office365 with OAuth (I'll be the first to admit that I haven't really dug into the DavMail docs)

(In reply to Rob Lemley [:rjl] from comment #156)

beta 4 is not built yet.

To clarify -- if you downloaded candidate build1 from the FTP server it does not include the patch in comment 152. I'm starting up build2 with the above patch shortly.

@Rob Lemley -- the version I just installed was from https://releases.mozilla.org/pub/thunderbird/candidates/78.0b4-candidates/build1/linux-x86_64/en-US/ dated June 26 ... Is that not a built version of 78.0b4? However as you mentioned, since it doesn't have the required patch that certainly explains why it won't work for me. I'll watch for build 2 and test it once it's posted.

In the meantime, is there some reason why my previous version of 79 no longer works? I'm getting confused as to what's what between these various builds, I normally just stick with the regular releases so I'm not familiar with the process here, but I thought the OAuth2 patches had been included in all the recent daily 79's?

Candidate builds may or may not actually be released depending on many factors. They're made available for QA and testing.

I'm not sure what might have happened to your 79 version unfortunately.

@Rob, thanks for the new build, but unfortunately I'm still seeing the same failure to connect to the exchange server as above.

I new thing though that might serve as a work-around for now... I was able to create a new account using an alias of my mailbox. It successfully found and connected to our exchange server, then I switched the authentication to OAuth2 and this generated a new Microsoft login screen where I could re-enter my credentials and it began downloading my emails into the new account. I was able to confirm the original mailbox was now working as well and deleted the new account. After restarting TB, my original mailbox is continuing to work.

So it seems the problem is that thunderbird is not popping up the MS login screen when the credentials expire, and not connecting automatically despite having selected for MS to remember my credentials. I'm not sure if the issue was caused while I was upgrading nightly versions, but there is apparently still an unhandled error remaining where the user is not prompted to sign in again.

If the server used is not outlook.office365.com, then I think all bets are off... that's something different. I don't understand how it would work for that case.

(In reply to Joe McCabe from comment #154)

I hear a "beep" and see an alert from Thunderbird Daily that tells me of an "Authentication error while connecting to server .xyz.com" (where ".xyz.com" is the domain from my email address and not the "outlook.office365.com" address specified in the account configuration).

Note that it's still possible that you get this (kind of correctly), in case IMAP is disabled for you account. When IMAP is disabled you can still do the OAuth2 dance but trying to use it you will get the message in the account wizard telling you authentication failed and that the configuration may be disabled for your account.

(In reply to Joe McCabe from comment #158)

(In reply to christophe.dubach from comment #157)

@Jeff: until the bug is sorted, you can use DavMail until this is sorted. It's far from perfect but at it more or less works ok.

Are there instructions that I've overlooked on how to make DavMail sit in front of Office365 with OAuth (I'll be the first to admit that I haven't really dug into the DavMail docs)

http://davmail.sourceforge.net/faq.html

It's working perfectly fine for me. The institution I worked at enabled MFA a few weeks ago and I've been using DavMail while waiting for Thunderbird's full support (I had Thunderbird working with the nightly build + some patches a while ago but decided I'll wait for a full stable release of Thunderbird before ditching DavMail).

@Magnus -- I was probably unclear above, but I think Joe McCabe and myself are both seeing the same thing. My email settings ARE pointing to outlook.office365.com, however the authentication error message that TB presents is giving an address similar to my employer's domain name ... essentially exchange.employer.com.

To make things even more confusing, sometimes when the MFA window comes up for me to enter my password, it is clearly the Microsoft page (it looks identical to the graphics and wording used when signing in the the web version of o365), but other times I have seen a prompt with my employer's name and terminology. Both of these screens have come up while only using outlook.office365.com as the receiving mail server.

the authentication error message that TB presents is giving an address similar to my employer's domain name ... essentially exchange.employer.com.

That is normal, because the auth error is probably generated in IMAP code, which doesn't know about the OAuth2 host. The "outlook.office365.com" host is hardcoded in the OAuth2 module code (in fact, that's what this bug added).
This message also makes sense to the end user, because he wanted to connect to exchange.employer.com, not Office365. You know about OAuth2, but normal users don't.

(In reply to Ben Bucksch (:BenB) from comment #166)

the authentication error message that TB presents is giving an address similar to my employer's domain name ... essentially exchange.employer.com.

That is normal, because the auth error is probably generated in IMAP code, which doesn't know about the OAuth2 host. The "outlook.office365.com" host is hardcoded in the OAuth2 module code (in fact, that's what this bug added).
This message also makes sense to the end user, because he wanted to connect to exchange.employer.com, not Office365. You know about OAuth2, but normal users don't.

Yeah, that's likely true (I see no way in the Office365 GUI to see if IMAP has been disabled by my employer), but when I trawl through the Mail.app PLIST I see one account referred to as "ews://{ID}" and several others referred to as "imap://{ID}", so it seems as though there's a good chance my employer forced the change away from IMAP when they began enforcing 2FA/OAuth.

Any chance TB will get EWS support?

Ews is on the chopping block with the rest of Basic Auth. Next year sometime.

(In reply to Jeremy Reeves from comment #168)

Ews is on the chopping block with the rest of Basic Auth. Next year sometime.

EWS isn't being deprecated. Basic Auth over EWS is. That said, it makes more sense to use the Graph REST API instead of EWS if you're going to explore HTTP-based protocols.

(In reply to Jesse Thompson from comment #169)

(In reply to Jeremy Reeves from comment #168)

Ews is on the chopping block with the rest of Basic Auth. Next year sometime.

EWS isn't being deprecated. Basic Auth over EWS is. That said, it makes more sense to use the Graph REST API instead of EWS if you're going to explore HTTP-based protocols.

You're absolutely right. Sorry about that! I totally pea brained that one. :)

(In reply to Magnus Melin [:mkmelin] from comment #140)

For this one it's taken a bit longer since there have been some issues where it's unclear where the problem lays.
Thunderbird 78.0 will be out for manual download in 10 days. It will take a bit longer before we auto-upgrade everybody.

Hi Magnus, Are we still on track for version 78 manual download this week? -Jeremy

With this implemented feature for O365 Oauth2 support, is it also possible to connect to a shared mailbox where a user has delegated access?

In the Microsoft Documentation I found this:
"In case of shared mailbox access using OAuth, application needs to obtain the access token on behalf of a user but replace the userName field in the SASL XOAUTH2 encoded string with the email address of the shared mailbox."
https://docs.microsoft.com/en-us/exchange/client-developer/legacy-protocols/how-to-authenticate-an-imap-pop-smtp-application-by-using-oauth#sasl-xoauth2-authentication-for-shared-mailboxes-in-office-365

Has this also been implemented?
I was trying it today but could not get it to work with Thunderbird 78.1.0.

Has anyone noticed that Thunderbird using Oauth2.0 #1) does not observe Conditional Access policy. e.g. I disconnect from my corporate VPN. All other clients disconnect after the 1 hour session timeout, but Thunderbird does not and 2) after I changed my account password, all other clients reauthenticate in the background using Modern Authentication but Thunderbird continues using the refresh token, no reauthentication to Azure is required.

I have a ticket open with Microsoft about this, because my Conditional Access policy should kick the client no more than 1 hour after disconnection.  So far the Azure guy says Exchange it letting Thunderbird continue with the existing token and is not getting reauthorization with Azure, which is where the Conditional Access policy lives. Still waiting for them to work with the Exchange folks to see whats up.

Just wondering if anyone else has observed/tried this in their environment assuming you have a similar policy.

(In reply to Jeremy Reeves from comment #173)

Has anyone noticed that Thunderbird using Oauth2.0 #1) does not observe Conditional Access policy. e.g. I disconnect from my corporate VPN. All other clients disconnect after the 1 hour session timeout, but Thunderbird does not and 2) after I changed my account password, all other clients reauthenticate in the background using Modern Authentication but Thunderbird continues using the refresh token, no reauthentication to Azure is required.

I have a ticket open with Microsoft about this, because my Conditional Access policy should kick the client no more than 1 hour after disconnection.  So far the Azure guy says Exchange it letting Thunderbird continue with the existing token and is not getting reauthorization with Azure, which is where the Conditional Access policy lives. Still waiting for them to work with the Exchange folks to see whats up.

Just wondering if anyone else has observed/tried this in their environment assuming you have a similar policy.

Yes, our Office 365 team noticed that Thunderbird appears as OWA (if I recall correctly) in the Conditional Access logs/policies. It may be due to the way the Thunderbird developers registered the app in Azure, perhaps. However, it wouldn't be surprising if fine-grained control of modern apps is a limitation of Conditional Access itself. Microsoft just released an update to CA that gives more granularity over clients, but Thunderbird still remains indistinct.

I think it is because Thunderbird's OAUTH token, registration and authflow is not using the Public Application OAUTH protocol MS defines with PCKE, etc. It is acting as a private/server centric application.

(In reply to Jeremy Reeves from comment #171)

...
Hi Magnus, Are we still on track for version 78 manual download this week? -Jeremy

We did in fact release 78.0 in mid-July

I filed bug 1668834 about the initial failure on setups, for which clicking the button once again will help.

(In reply to Jesse Thompson from comment #174)

(In reply to Jeremy Reeves from comment #173)

Has anyone noticed that Thunderbird using Oauth2.0 #1) does not observe Conditional Access policy. e.g. I disconnect from my corporate VPN. All other clients disconnect after the 1 hour session timeout, but Thunderbird does not and 2) after I changed my account password, all other clients reauthenticate in the background using Modern Authentication but Thunderbird continues using the refresh token, no reauthentication to Azure is required.

I have a ticket open with Microsoft about this, because my Conditional Access policy should kick the client no more than 1 hour after disconnection.  So far the Azure guy says Exchange it letting Thunderbird continue with the existing token and is not getting reauthorization with Azure, which is where the Conditional Access policy lives. Still waiting for them to work with the Exchange folks to see whats up.

Just wondering if anyone else has observed/tried this in their environment assuming you have a similar policy.

Yes, our Office 365 team noticed that Thunderbird appears as OWA (if I recall correctly) in the Conditional Access logs/policies. It may be due to the way the Thunderbird developers registered the app in Azure, perhaps. However, it wouldn't be surprising if fine-grained control of modern apps is a limitation of Conditional Access itself. Microsoft just released an update to CA that gives more granularity over clients, but Thunderbird still remains indistinct.

Ive been working on and off with Microsoft to try to hunt this issue down. Microsoft just today confirmed that the registered Thunderbird app in Azure is behaving as a "confidential client", which is meant for hardened servers and generally hard to access systems. A "Public client" would be for regular desktops clients or web browser not trusted to keep application secrets.
https://docs.microsoft.com/en-us/azure/active-directory/develop/msal-client-applications

The escalation engineer is not clear on why this app is configured as a confidential client since it is an email client, and recommended that I disable access to the app in my tenant until we have a clear and valid reason this app requires to be configured as confidential client instead of a public client app.
https://docs.microsoft.com/en-us/azure/active-directory/develop/msal-client-application-configuration

It would seem that by using confidential client, the Thunderbird client is able to capture and store confidential secrets (refresh token, for instance), that could in effect be captured using fiddler or other snooping tools. In effect, the user is never asked for credentials again because it has the refresh token which is essentially a password for the application to continue access to the mailbox, even surviving account password changes, which traditionally should NOT be the case. Password changes should require a new authentication, but this method does not require it.

My assumption is the Public client would not behave the same way, and probably stores only temporary credentials. My hope is that the app can be reconfigured for Public client and behave normally. :)

@mkmelin, Are you able to comment if the app configuration in Azure is infact a confidential client? Is there a reason it cannot be configured as a public client? Can we test a Public client instance of a Thunderbird app in a beta?

Jeremy

Is this two-factor authentication as the title says or oAth? I for one am confused as all the discussion appears to revolve around oAth issues and I see basically nothing about two-factor authentication (those messages that appear on your phone with a login token.) that Thunderbird does not support and folks keep asking for because they think that is how it has to be done.

(In reply to Matt from comment #179)

Is this two-factor authentication as the title says or oAth? I for one am confused as all the discussion appears to revolve around oAth issues and I see basically nothing about two-factor authentication (those messages that appear on your phone with a login token.) that Thunderbird does not support and folks keep asking for because they think that is how it has to be done.

Hi Matt. In order to support MFA with O365, OAuth support had to be implemented/fixed to work with O365. One that was implemented, O365 MFA could work. In doing so, however, the Azure App that makes this possible may be configured in a way that gives the azure app too much permissions and bypasses many administrative restrictions. That is what my comment today is about by reviving this discussion. Before filing a separate bug report, I wanted to keep it tied into the original bug fix for additional discussion.

(In reply to Jeremy Reeves from comment #178)

It would seem that by using confidential client, the Thunderbird client is able to capture and store confidential secrets (refresh token, for instance), that could in effect be captured using fiddler or other snooping tools. In effect, the user is never asked for credentials again because it has the refresh token which is essentially a password for the application to continue access to the mailbox, even surviving account password changes, which traditionally should NOT be the case. Password changes should require a new authentication, but this method does not require it.

My assumption is the Public client would not behave the same way, and probably stores only temporary credentials. My hope is that the app can be reconfigured for Public client and behave normally. :)

This is fascinating, and Thunderbird probably sits somewhere between the two client types (Public vs Confidential). A mail application running on a user's desktop PC is in many ways very different than a browser-based application in that it has an out-of-band storage (hopefully secure) mechanism where secrets are stored. And by definition, there is a "configuration" step, a one-off process where a user explicitly adds an account and (usually) authenticates. Whether a password change (in O365) should force a reauthentication in Thunderbird is open for debate, but certainly in the absence of a password or other critical configuration change, the user should "never" be prompted for credentials (the whole OAuth flow) again.

Just a few extra thoughts...

(In reply to Adam Batkin from comment #181)

(In reply to Jeremy Reeves from comment #178)

It would seem that by using confidential client, the Thunderbird client is able to capture and store confidential secrets (refresh token, for instance), that could in effect be captured using fiddler or other snooping tools. In effect, the user is never asked for credentials again because it has the refresh token which is essentially a password for the application to continue access to the mailbox, even surviving account password changes, which traditionally should NOT be the case. Password changes should require a new authentication, but this method does not require it.

My assumption is the Public client would not behave the same way, and probably stores only temporary credentials. My hope is that the app can be reconfigured for Public client and behave normally. :)

This is fascinating, and Thunderbird probably sits somewhere between the two client types (Public vs Confidential). A mail application running on a user's desktop PC is in many ways very different than a browser-based application in that it has an out-of-band storage (hopefully secure) mechanism where secrets are stored. And by definition, there is a "configuration" step, a one-off process where a user explicitly adds an account and (usually) authenticates. Whether a password change (in O365) should force a reauthentication in Thunderbird is open for debate, but certainly in the absence of a password or other critical configuration change, the user should "never" be prompted for credentials (the whole OAuth flow) again.

Just a few extra thoughts...

Oauth does not mean never reauthenticate. Oauth also does not mean SSO. Oauth is an authentication protocol that offers the opportunity to utilize SSO, and in some cases would mean the user does not have to authenticate themselves because it’s already been done through some other means.

The issue I see here is Tbird is behaving like a service or service principal working on behalf of the account, and not behaving like a client allowing the user to act for itself.

I know Tbird would not work this way, but let’s consider OWA with ADFS as the IdP. After a period of time(session expires) or password change, a user token is no longer valid and it is required to reauth through the IdP to refresh. Sure this is usually SSO with cached credentials, but it’s still a reauth as the account. Outlook does the same thing using Windows credentials with Modern Auth. Tbird is bypassing that requirement by reauthing through the azure app on behalf of the user as a service, not allowing the user to reauthenticate themselves with the new credential.

And because it’s acting like a service, Microsoft is letting it squeak past the security measure in place leading us to this active issue. I’m going to have a couple hundred angry folks later this year if it can’t be resolved because when Basic Auth is finally killed, we likely will not allow them to use Tbird with Oauth in the current state. Trying to avoid this scenario.

Thunderbird is absolutely not a confidental client, it cannot maintain the secrecy of the globally issued client secret (it is currently published for all to see in the github). It should be using the public client type, with no client secret, using PCKE instead. Apps that can't maintian the client secret can't have app-specific security policies applied because basically anyone can forge the authentication source.

(In reply to Jason Gunthorpe from comment #183)

Thunderbird is absolutely not a confidental client, it cannot maintain the secrecy of the globally issued client secret (it is currently published for all to see in the github). It should be using the public client type, with no client secret, using PCKE instead. Apps that can't maintian the client secret can't have app-specific security policies applied because basically anyone can forge the authentication source.

I don't disagree with you there.

Magnus, Can this be revisited?

(In reply to Jeremy Reeves from comment #173)

Has anyone noticed that Thunderbird using Oauth2.0 #1) does not observe Conditional Access policy. e.g. I disconnect from my corporate VPN. All other clients disconnect after the 1 hour session timeout, but Thunderbird does not and 2) after I changed my account password, all other clients reauthenticate in the background using Modern Authentication but Thunderbird continues using the refresh token, no reauthentication to Azure is required.

I have a ticket open with Microsoft about this, because my Conditional Access policy should kick the client no more than 1 hour after disconnection.  So far the Azure guy says Exchange it letting Thunderbird continue with the existing token and is not getting reauthorization with Azure, which is where the Conditional Access policy lives. Still waiting for them to work with the Exchange folks to see whats up.

Just wondering if anyone else has observed/tried this in their environment assuming you have a similar policy.

And additional issue I have noticed recently in multiple o365 School / corporate environments is that the Thunderbird client does not provide the required "DeviceID" value from the Azure Primary Refresh Token. The device ID value is used to determine authorization for Conditional Access rules based on device state or compliance.

As a result any organizations which use Conditional Access policies which referrence the DeviceID - the Thunderbird client will be automatically blocked from connecting (Even if connecting from a compliant device) due to the required value not being provided in the Auth process. Users will instead get a pop up which states:

This application contains sensitive information and can only be accessed from: Devices and Client applications that meet <organizations> management compliance policy.

Under the "more details" section of the client security popup the DeviceID information will show as blank indicating this information was not provided by the Thunderbird client. Similarly the O365 signin logs will show this information was not provided by the client and you will see an explicit failure in the Conditional Access section showing the rule could not be evaluated as the DeviceID was not provided.

You can pull this information on a windows machine using the cmd "dsregcmd /status". Its also available from AD joined MAC machines. o365 OAuth process clients are meant to provide this information when available.

(In reply to Jeremy Reeves from comment #182)

(In reply to Adam Batkin from comment #181)

(In reply to Jeremy Reeves from comment #178)

It would seem that by using confidential client, the Thunderbird client is able to capture and store confidential secrets (refresh token, for instance), that could in effect be captured using fiddler or other snooping tools. In effect, the user is never asked for credentials again because it has the refresh token which is essentially a password for the application to continue access to the mailbox, even surviving account password changes, which traditionally should NOT be the case. Password changes should require a new authentication, but this method does not require it.

My assumption is the Public client would not behave the same way, and probably stores only temporary credentials. My hope is that the app can be reconfigured for Public client and behave normally. :)

This is fascinating, and Thunderbird probably sits somewhere between the two client types (Public vs Confidential). A mail application running on a user's desktop PC is in many ways very different than a browser-based application in that it has an out-of-band storage (hopefully secure) mechanism where secrets are stored. And by definition, there is a "configuration" step, a one-off process where a user explicitly adds an account and (usually) authenticates. Whether a password change (in O365) should force a reauthentication in Thunderbird is open for debate, but certainly in the absence of a password or other critical configuration change, the user should "never" be prompted for credentials (the whole OAuth flow) again.

Just a few extra thoughts...

Oauth does not mean never reauthenticate. Oauth also does not mean SSO. Oauth is an authentication protocol that offers the opportunity to utilize SSO, and in some cases would mean the user does not have to authenticate themselves because it’s already been done through some other means.

The issue I see here is Tbird is behaving like a service or service principal working on behalf of the account, and not behaving like a client allowing the user to act for itself.

I know Tbird would not work this way, but let’s consider OWA with ADFS as the IdP. After a period of time(session expires) or password change, a user token is no longer valid and it is required to reauth through the IdP to refresh. Sure this is usually SSO with cached credentials, but it’s still a reauth as the account. Outlook does the same thing using Windows credentials with Modern Auth. Tbird is bypassing that requirement by reauthing through the azure app on behalf of the user as a service, not allowing the user to reauthenticate themselves with the new credential.

And because it’s acting like a service, Microsoft is letting it squeak past the security measure in place leading us to this active issue. I’m going to have a couple hundred angry folks later this year if it can’t be resolved because when Basic Auth is finally killed, we likely will not allow them to use Tbird with Oauth in the current state. Trying to avoid this scenario.

Is this still an issue, and does anyone know if it is being actively worked on?

(In reply to karapwalsh from comment #187)

Is this still an issue, and does anyone know if it is being actively worked on?

Yes it’s still an issue. No it doesn’t appear to be worked on. I submitted a separate bug 1685414 last year. With Microsoft’s renewed one line to deprecate IMAP Basic Auth beginning Oct 2022, my users are getting anxious and I’m preparing to deliver the news that they will not be able to use TB if it’s not resolved before then. (Need adequate time to migrate users to Outlook or other software that supports the way we expect.)

(In reply to Jeremy Reeves from comment #188)

(In reply to karapwalsh from comment #187)

Is this still an issue, and does anyone know if it is being actively worked on?

Yes it’s still an issue. No it doesn’t appear to be worked on. I submitted a separate bug 1685414 last year. With Microsoft’s renewed one line to deprecate IMAP Basic Auth beginning Oct 2022, my users are getting anxious and I’m preparing to deliver the news that they will not be able to use TB if it’s not resolved before then. (Need adequate time to migrate users to Outlook or other software that supports the way we expect.)

The bug is marked as fixed, because the developers have fixed it. So not it is not being worked on. For those that find the solution used in Thunderbird unacceptable to them. There is always https://addons.thunderbird.net/en-US/thunderbird/addon/owl-for-exchange/?src=ss As the addon does not use IMAP or SMTP, it can bypass this bug, and therefore also it's fix.

Basically I think discussion here is moot as the bug is closed.

Unfortunately, owl-for-exchange is no longer free. They now charge 10€ per year:
https://www.beonex.com/owl/

It worked well but I unfortunately cannot afford the extra cost. If only Mozilla could implement this natively.

We made it public client in bug 1685414 (new app id required). You can try it in beta.