User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:86.0) Gecko/20100101 Firefox/86.0

Steps to reproduce:

Our company recently moved to the Microsoft Office365 GCC High environment for our email. In anticipation of Microsoft disabling "Basic Authentication" later this year, we are testing the use of OAUTH2 authentication. So, I set my account settings/<office365.us - account/server settings/authentication method to OAUTH2.

We have contacted Microsoft Support. They have indicated that Thunderbird must be registered in the Azure AD for us Gov. cloud. See the following:

https://docs.microsoft.com/en-us/azure/active-directory/develop/authentication-national-cloud

Microsoft indicated that Thunderbird has registered with the "commercial" AD cloud.

Actual results:

Thunderbird either doesn't respond -or- responds with message saying roughly that the selected authentication mechanism is not supported.

Expected results:

Thunderbird should have contacted our OAUTH provider which would have followed on with the expected multifactor authentication.

I'd be happy to provide any additional info from our Microsoft support ticket and additional help and information as needed.

I'm also seeing this behavior using the O365 Gov endpoints, I logged debug output and I can see Microsoft is sending AUTH=XOAUTH2 as an option but the Thunderbird client never attempts it.

As a test I tried switching the server to outlook.office365.com and it opened the ADFS sign in page for my organization. The Azure consent page is for foreign cloud application as expected since I'm using the wrong endpoint. I've tried creating an app registration in Azure for Thunderbird but that seems to be irrelevant at this point.

DEBUG LOG:
2021-05-05 15:00:41.847000 UTC - [(null) 38224: IMAP]: I/IMAP 000002A41FE2B800:outlook.office365.us:NA:CreateNewLineFromSocket: * CAPABILITY IMAP4 IMAP4rev1 AUTH=PLAIN AUTH=XOAUTH2 SASL-IR UIDPLUS ID UNSELECT CHILDREN IDLE NAMESPACE LITERAL+

2021-05-05 15:00:41.847000 UTC - [(null) 38224: IMAP]: D/IMAP SetConnectionStatus(0x0)
2021-05-05 15:00:41.855000 UTC - [(null) 38224: IMAP]: D/IMAP ReadNextLine [rv=0x0 stream=000002A428A5BC10 nb=28 needmore=0]
2021-05-05 15:00:41.855000 UTC - [(null) 38224: IMAP]: I/IMAP 000002A41FE2B800:outlook.office365.us:NA:CreateNewLineFromSocket: 1 OK CAPABILITY completed.

2021-05-05 15:00:41.855000 UTC - [(null) 38224: IMAP]: D/IMAP SetConnectionStatus(0x0)
2021-05-05 15:00:41.855000 UTC - [(null) 38224: IMAP]: D/IMAP Try to log in
2021-05-05 15:00:41.855000 UTC - [(null) 38224: IMAP]: D/IMAP IMAP auth: server caps 0x800087635, pref 0x0, failed 0x0, avail caps 0x0
2021-05-05 15:00:41.855000 UTC - [(null) 38224: IMAP]: D/IMAP (GSSAPI = 0x1000000, CRAM = 0x20000, NTLM = 0x100000, MSN = 0x200000, PLAIN = 0x1000, LOGIN = 0x2, old-style IMAP login = 0x4, auth external IMAP login = 0x20000000, OAUTH2 = 0x800000000)
2021-05-05 15:00:41.855000 UTC - [(null) 38224: IMAP]: D/IMAP No remaining auth method
2021-05-05 15:00:41.891000 UTC - [(null) 38224: IMAP]: E/IMAP login failed entirely
2021-05-05 15:00:41.894000 UTC - [(null) 38224: IMAP]: D/IMAP SetConnectionStatus(0x80004005)

We are also seeing this behavior with Thunderbird and O365 GCC High. We had no issues with standard O365 using OAuth2. Microsoft GCC High customer support did a "fiddler" trace and claim there is a caching issue with Thunderbird. Their response below:

{

HTTP/200 responses are cacheable by default, unless Expires, Pragma, or Cache-Control headers are present and forbid caching.
Legacy Pragma Header is present: no-cache

HTTP/1.1 Cache-Control Header is present: no-store,no-cache
no-store: This response MUST NOT be stored in a cache.

HTTP/1.1 Vary Header is present: Accept-Encoding
The cache MUST contact the server to verify freshness unless the value of the headers named match those of the request that generated the cache entry.

Note: IE has limited support for Vary. See http://fiddler2.com/r/?ievary

!! WARNING: Responses which VARY should specify an ETAG to enable conditional revalidation requests.
This response contains neither an ETAG nor a Last-Modified time. This will prevent a Conditional Revalidation of this response.

This HTTP/304 response indicates that the existing cached response remains fresh. Cache-lifetime headers on a HTTP/304 response may be used to update the cached response's freshness.

Under RFC2616, HTTP/403 responses will not be cached regardless of what caching headers may be present.
This response does not specify explicit HTTP Cache Lifetime information and does not specify a Last-Modified date. Heuristic expiration is typically based on Last-Modified date. Lacking Last-Modified, this response may be revalidated on every use or once per browsing session, depending on the browser configuration.

This response contains neither an ETAG nor a Last-Modified time. This will prevent a Conditional Revalidation of this response.

================
Learn more about caching at http://fiddler2.com/r/?httpperf

I am going to get Azure involved with this since this is a third party app we are suing to connect to the service. They should hopefully be able to tell us the cause of the cashing problem that is being experienced.

Appreciate your patience on this.

HTTP/1.0 Expires Header is present: Tue, 25 Jan 2022 20:42:43 GMT

HTTP/1.1 Cache-Control Header is present: no-cache,max-age=3600
no-cache: This response MUST NOT be reused without successful revalidation with the origin server.
max-age: This resource will expire in 1 hours. [3600 sec]

HTTP/1.1 ETAG Header is present: "BjAwNAeiNkudMgjQF1LB5Pbu6nV7T4yYcMKgObt39Hs="

================
Learn more about caching at http://fiddler2.com/r/?httpperf
}

Out of curiosity I installed Thunderbird 91 in the hopes that maybe something was changed in a later build but just confirming this is still an issue 1 year later.

Just ran into this as we are progressing with our GCC High migration. This is really distressing.

We received a notice from Microsoft about the plans to disable basic auth for GCCH on March 31st, 2023. If Mozilla doesn't make a change to the endpoints (or at least offer a way to manually enter them in), we'll have to stop supporting Thunderbird as a mail client since it will no longer function.

It seems to me that once TB registered with GCC High, the following patch with the appropriate ID/secret would be all that is needed?
diff -up thunderbird-91.11.0/comm/mailnews/base/src/OAuth2Providers.jsm.gcchigh thunderbird-91.11.0/comm/mailnews/base/src/OAuth2Providers.jsm
--- thunderbird-91.11.0/comm/mailnews/base/src/OAuth2Providers.jsm.gcchigh 2022-06-27 20:40:22.000000000 -0600
+++ thunderbird-91.11.0/comm/mailnews/base/src/OAuth2Providers.jsm 2022-07-19 12:45:05.243353234 -0600
@@ -60,6 +60,20 @@ var kHostnames = new Map([
"https://outlook.office365.com/IMAP.AccessAsUser.All https://outlook.office365.com/POP.AccessAsUser.All https://outlook.office365.com/SMTP.Send offline_access",
],
],

• [

• "outlook.office365.us",

• [

•  "login.microsoftonline.us",
  

•  "https://outlook.office365.us/IMAP.AccessAsUser.All https://outlook.office365.us/POP.AccessAsUser.All https://outlook.office365.us/SMTP.Send offline_access",
  

• ],

• ],

• [

• "smtp.office365.us",

• [

•  "login.microsoftonline.us",
  

•  "https://outlook.office365.us/IMAP.AccessAsUser.All https://outlook.office365.us/POP.AccessAsUser.All https://outlook.office365.us/SMTP.Send offline_access",
  

• ],

• ],

  // For testing purposes.
  ["mochi.test", ["mochi.test", "test_scope"]],
  @@ -134,6 +148,17 @@ var kIssuers = new Map([
  ],
  ],

• [

• "login.microsoftonline.us",

• [

•  "XXX", // Application (client) ID
  

•  "XXX", // @see App registrations | Certificates & secrets
  

•  // https://docs.microsoft.com/en-us/azure/active-directory/develop/active-directory-v2-protocols#endpoints
  

•  "https://login.microsoftonline.us/common/oauth2/v2.0/authorize",
  

•  "https://login.microsoftonline.us/common/oauth2/v2.0/token",
  

• ],

• ],

• // For testing purposes.
  [
  "mochi.test",

Your comment got mangled - maybe you can update it (surround the code section with a pair of 3 single quotes).
Are you saying outlook.office365.us/login.microsoftonline.us is to be used for some environments?

This Microsoft docs page is network focused for allowing access to GCC High IP addresses, but it does also list the DNS names of the various endpoints:
https://docs.microsoft.com/en-us/microsoft-365/enterprise/microsoft-365-u-s-government-gcc-high-endpoints?view=o365-worldwide

The default commercial O365 endpoints end with .com but for GCC High they usually end with .us. Then GCC DoD has its own set of endpoint names.

I've started testing with the attached patch, and am close. I setup an app registration in my GCC High Azure portal (I've redacted the ID and secret in the patch) - I needed to make it multi-tenant and set the redirect URL to http://localhost to match the TB usage but still hit an error. Recompiling with the latest change to try to get autodiscovery to work and to allow configuring OAuth2 in the GUI.

Yes, GCC High uses office365.us and login.microsoftonline.us instead of the .com versions used by the Commercial cloud (and "regular" GCC).

I'm not sure what it would take for Mozilla to register the Thunderbird app with GCC High Azure (https://portal.azure.us). I've reached out to our vendor to see if they can find out.

If TB allowed for customization of OAuth2 configs we could work around it that way, but alas it does not.

With the latest patch I can configure OAuth2 in the GUI, but the account verification still fails. It pops up the OAuth2 window, I enter the password, then it closes.

(In reply to Orion Poplawski from comment #9)

Created attachment 9286263 [details] [diff] [review]
thunderbird-gcchigh-redacted.patch

I've started testing with the attached patch, and am close. I setup an app registration in my GCC High Azure portal (I've redacted the ID and secret in the patch) - I needed to make it multi-tenant and set the redirect URL to http://localhost to match the TB usage but still hit an error. Recompiling with the latest change to try to get autodiscovery to work and to allow configuring OAuth2 in the GUI.

Yes, GCC High uses office365.us and login.microsoftonline.us instead of the .com versions used by the Commercial cloud (and "regular" GCC).

I'm not sure what it would take for Mozilla to register the Thunderbird app with GCC High Azure (https://portal.azure.us). I've reached out to our vendor to see if they can find out.

If TB allowed for customization of OAuth2 configs we could work around it that way, but alas it does not.

I work with Doug and Mabry. Mabry meant to say that this is happening with our GCC-H tenant as well. Is there anything we could do to help facilitate testing this fix?

Thanks,
Q

(In reply to Mabry Tyson from comment #12)

(In reply to Orion Poplawski from comment #9)

Created attachment 9286263 [details] [diff] [review]
thunderbird-gcchigh-redacted.patch

I've started testing with the attached patch, and am close. I setup an app registration in my GCC High Azure portal (I've redacted the ID and secret in the patch) - I needed to make it multi-tenant and set the redirect URL to http://localhost to match the TB usage but still hit an error. Recompiling with the latest change to try to get autodiscovery to work and to allow configuring OAuth2 in the GUI.

Yes, GCC High uses office365.us and login.microsoftonline.us instead of the .com versions used by the Commercial cloud (and "regular" GCC).

I'm not sure what it would take for Mozilla to register the Thunderbird app with GCC High Azure (https://portal.azure.us). I've reached out to our vendor to see if they can find out.

If TB allowed for customization of OAuth2 configs we could work around it that way, but alas it does not.

I think I'm configuring the Azure application incorrectly. (The error console is a red-herring - same errors there with a successful login). The server is responding:

AADSTS700025: Client is public so neither 'client_assertion' nor 'client_secret' should be presented.

I'm not entirely sure what is meant by "public" here. If I switch the app from multi-tenant to single-tenant I get:

Application 'aa257950-6a26-40ec-874e-ca637a48ccc2'(NWRA Thunderbird) is not configured as a multi-tenant application. Usage of the /common endpoint is not supported for such applications created after '10/15/2018'. Use a tenant-specific endpoint or configure the application to be multi-tenant.

It also seems likes TB's OAuth2 process isn't correct - what is it doing using a client_secret that clearly isn't secret?

client_secret isn't really secret in the OAuth2 spec, in the normal flow.

As of July 12, 2022, our GCC-H tenant is no longer allowing Thunderbird to access our Office365.us email. Microsoft has not confirmed that "Basic Authentication" for our environment has been disabled, but everything appears to be indicating that this is what has happened. This issue has affected all IMAP/(Basic Auth) access to Office365.us which includes client programs that are not Thunderbird.

This has left a great deal or our user base in a horrible position with respect to email. Our Linux based users have only the Outlook Web Access as an option for accessing their email.

I submitted this bug report over a year ago. I do not seem to have the power to affect the priority. Is there anything else I can do?

Could someone from Mozilla please respond? It seems like this should be a fairly simple thing to fix and it's causing us a lot of grief. Thank you

(In reply to Orion Poplawski from comment #18)

Could someone from Mozilla please respond? It seems like this should be a fairly simple thing to fix and it's causing us a lot of grief. Thank you

Check the links. It looks to me your responses are coming from about as far up the Thunderbird food chain as can be.
https://wiki.mozilla.org/Modules/Thunderbird
https://fi.linkedin.com/in/magnusmelin

Will be looking at it soon.

(In reply to Magnus Melin [:mkmelin] from comment #20)

Will be looking at it soon.

Thank you. I'm happy to help test or in any other way if needed.

Would it help if we were able to help fund development of this?

We can maybe look at this once the configuration and patch from bug 1685414 is confirmed working and on beta at least. If I'm understanding this correctly, the patch for this will just have a configuration similar to the one added in that bug, and we'll need a registration on https://portal.azure.us.(In reply to Orion Poplawski from comment #15)

It also seems likes TB's OAuth2 process isn't correct - what is it doing using a client_secret that clearly isn't secret?

Correct, there isn't supposed to be a client_secret and the new configuration has none.

Bug 1685414 is now marked as fixed, and in the nightly builds. I thought I'd see if this bug (auth thru Office365.us) is fixed by that
It wasn't clear to me whether there's a bit more that needs to be done for this to work..

At least for me, with the most recent Daily, I am not able to log into the Office365.us to get mail. For all II know this may be peculiar to our site. I hope others will try it.

I installed Thunderbird Daily 108.0a1 (2022-10-23) (64-bit) for Mac and proceeded to set it up for our site. (I have no control over the Microsoft aspects of our site). During "Set up your existing email address", I entered first.last@example.com + name + password.

When I hit Continue, I got a pop-up "Dally tound your account setup intormation on ottice36b.us. Do you want
to proceed and submit your credentials?" I hit Login.

Now, TB Daily got confused because we apparently have a mail.example.com that answers on ports 143, 587, 993, but is not really a mail server for us. It offered a configuration using that host, but that is wrong. I had to do a manual configuration with IMAP on port 993/TLS with Normal Password on outlook.office365.us and Submission on port 587/StartTLS with Normal Password on the same host. (Those settings match what is in OWA's Settings > Mail > Sync Email)

Now, here's our potentially site specific issue: When I connect to https://outlook.office365.us/mail, another site (example.okta.com) asks for my user id (E12345) and password. When I give that, I can then find myself back at outlook.office365.us/mail and able to read my mail.

In the account configuration, I tried these usernames: E12345 (what I expect), first.last@example;.com, E12345@example.com. None of them worked.

Years back, I recall being able to trace through IMAP authentication in TB, but I couldn't easily find how to turn that debugging on again. If someone can point me to that, I can dig deeper into the authentication process to see where it breaks down.

For imap logging, see https://wiki.mozilla.org/MailNews:Logging
For OAuth2 debugging, set mailnews.oauth.loglevel to "All"

Is this really expected to be working? It's certainly not automatic. I tried with my @nwra.onmicrosoft.us email addres and it failed to find the settings for my email account. If I manually put in outlook.office365.us / smtp.office365.us and re-test it doesn't give my the option of OAuth2. If I do Advanced, then I can select OAuth2 for the IMAP server, but not for the smtp server. But in the Activity Manager, I see "The IMAP server outlook.office365.us does not support the selected authentication method." and it can't load anything and doesn't prompt me to login.

I would have thought you would have at least needed to add the OAuth2 providers in mailnews/base/src/OAuth2Providers.jsm, which I don't see. Or is there some new method to detect OAuth2 configuration?

(In reply to Mabry Tyson from comment #24)
Well at least you need to set Thunderbird to use OAuth2 for authentication, not Normal Password.

(In reply to Orion Poplawski from comment #26)
Unlikely to work yet. Needs a separate registration for outlook.office365.us

(In reply to Magnus Melin [:mkmelin] from comment #27)

(In reply to Orion Poplawski from comment #26)
Unlikely to work yet. Needs a separate registration for outlook.office365.us

Do we have any kind of estimate as to when this might be done? Thank you.

(In reply to Magnus Melin [:mkmelin] from comment #27)

(In reply to Mabry Tyson from comment #24)
Well at least you need to set Thunderbird to use OAuth2 for authentication, not Normal Password.

(In reply to Orion Poplawski from comment #26)
Unlikely to work yet. Needs a separate registration for outlook.office365.us

Can anything be done on our part to expedite the outlook.office3365.us registration?

Our company has initiated a $50/mo ongoing contribution to TB in thanks for the work done to make TB a great cross-platform email tool for our company, and to hopefully provide some more resources to tackle issues like these going forward. Unfortunately lack of GCC High support has been a big burden on our company.

So, we seem to be making some progress. With 102.7.2 I can configure IMAP for outlook.office365.us with OAuth2 and it works! However, account autoconfigure sets up SMTP to be outlook.office365.us (no idea if this is really correct or not, but smtp.office365.us is an alias for it, so I think it is) but does not allow me to select OAuth2 authentication with it (or smtp.office365.us if I change the server name to that).

Ar you sure that works? I'm don't think it's possible, since outlook.office365.us isn't listed as a host in https://searchfox.org/comm-central/rev/42e044a2814aba126d0c080619e2ee247a37c39c/mailnews/base/src/OAuth2Providers.jsm#51

Re autoconfig, you can set mail.setup.loglevel to All to figure out where that value is coming from.

Has there been any progress on this? This is really hampering our organization from effectively using thunderbird.

I would be willing to throw some money at getting GCC High support. Also, my org and I are willing to test fixes for this.

One problem we have is that users of Thunderbird don't get prompted to ask the administrator to allow access to the app when logging in with OAuth2. We'd usually expect to see "This is the first time this app is trying to access your tenant (or whatever), please ask your admin to allow it" And the application would appear on the admin portal to be approved. This doesn't seem to be the case with GCC High. Not sure if the Moz folks need to submit another application for GCC High or what.

After following this bug regarding a change in the way OAuth needs to be handled for protecting client secrets https://bugzilla.mozilla.org/show_bug.cgi?id=1685414 I tried having our admin manually register a patched nightly build with the portal. Oauth2 would succeed, but when presenting the token to the imap server the imap server would not consider the token valid. I'm not sure how to get better logs of that. If I should open a new ticket for that I will.

Please advise