Is anyone still seeing this with Taskcluster ? It seems to have stopped for me at some point, currently using 66.0 betas.

let's say "no" and reopen if wrong

I'm still seeing this unfortunately :(

Prior to the taskcluster switchover on November 9, 2019 I used to have treeherder log me out every Monday, like clockwork.

Post-deployment of the new taskcluster, I am seeing logoffs often in matter of days, sometimes even hours.

I am using containers for all of my treeherder tabs, on Firefox 72.0a1, maosx1014 and macosx1015.

Are treeherder, https://firefox-ci-tc.services.mozilla.com, and auth0 all in the same container?

Yes - my typical environment has 3 different containers running, each loaded with treeherder, taskcluster and auth0.

Say I take the personal container:

• open treeherder
• log in
• prompted for auth0 if required
• approve via duo

Hm, if everything is in the same container, then this should work "normally", like a regular browser session, right? The cases where we've seen issues with containers are because when a token renewal occurred, the service being called to check the renewal wasn't active in that container.

So I'm guessing there's some service that's pinned to a different container, or that something in the process of opening new tabs for the sign-in process "loses track" of what container it's operating in.

Yes, the container should work like normal browser sessions, just separated out so from the perspective of treeherder there should be 3 separate sessions for user egao when I use 3 containers. I am not familiar with the technical implementation of how containers work, so I cannot comment on that portion.

Note, I only use containers to separate out visually the work I am doing, so I'm not relying on some intrinsic behavior of the container itself for my work. I could replace my workflow with a tab group and it would do what I would like it to.

(In reply to Dustin J. Mitchell [:dustin] (he/him) from comment #17)

Are treeherder, https://firefox-ci-tc.services.mozilla.com, and auth0 all in the same container?

Yes for me too (and I only have one container where they are opened, and that's the only container I use other than the default). Maybe there is some intermediary domain which is not pinned to the container and so is opened in the default container.

Are there tools in Firefox that help debug container issues like this? This must be fairly common..

(In reply to Dustin J. Mitchell [:dustin] (he/him) from comment #22)

Are there tools in Firefox that help debug container issues like this? This must be fairly common..

Andrea, maybe you know? IIRC you worked on containers.

jmaher also noticed the issue this week as did Andreea Pavel. At least the latter doesn't use containers.

Andrea, maybe you know? IIRC you worked on containers.

Redirect to jkt. I use gdb/rr...

OK, sounds like containers are off the hook (or we're dealing with two issues..).

Can someone provide a reproduction recipe?

As an update, this automatic logoff also occurs for Bugzilla and Treeherder, so is not an issue limited to Taskcluster.

There is no reliable step to reproduce, since it seems to occur out of the blue.

1. log in and auth0 at all locations as required (treeherder, taskcluster, bugzilla, etc)
2. use container for the day
3. put computer to sleep
4. wake up next morning
5. refresh tabs for treeherder, taskcluster, etc.

With some probability, user will be logged out of some or all of the services.

I am not experiencing a logout from Bugzilla. This morning, I logged into the firefox-ci cluster to attempt to retrigger a hook and later into Treeherder to request the task. At least 7h later, TH showed me as logged out. Not interacting with the tools into which one logged in or using the permissions associated with them might be relevant. (Had seen a logoff on another day ~6h after login).

Each sheriff experienced 3 logouts since the beginning of our shift, and as far we can tell, the logouts were made after about every 2h. As you can see in the screenshot attached, Treeherder showed first the bug as associated with the failure and immediately the error after that, the failure remaining unclassified.
Strangely, Treeherder page showed us as being still logged in, the name was visible in the top right hand corner until the refresh of the page after which we we had to log back in from the Login/Register button.

Comment 28 was about a logout from Treeherder. I don't think Treeherder shows whether a user is "logged in" to Taskcluster or not. Overall, I think there are still a number of reports of what might be different bugs, all without enough detail to debug. If we're going to get to the bottom of this, I think we'll need a reproduction recipe, ideally starting with an Auth0 login in a fresh session. Probably starting a browser with a fresh profile would help with the "not interacting with the tools" that aryx mentioned, too.

1. Opened Treeherder in a new profile
2. Logged in
3. Let it idle for 7h.
4. Tab still showed me as logged.
5. Reloaded the tab.
6. Treeherder showed me as not logged in.

The current and the previous reported that each got backed out "a couple of times" but not during their previous shift (2.5 days ago).

Related to the Django upgrade?

This sounds like an auth0/TH backend issue that doesn't anything to do with Taskcluster credentials. Those credentials are separately retrieved and are independent from each other as of the Nov 9th changeover. When you log in to Treeherder, your moz credentials are being retrieved and we're doing some session management via our django backend (classifying failures involves savings changes to our database but wouldn't have anything to do with taskcluster specifically, which is what this original bug was for).

I don't think the existing TH/auth0/session management logic was changed too much for the Nov 9th changeover, but I'll take a look. From bug 1605651 and Edwin's comments, this seems like it's been happening for long enough that I'm not sure the recent Django upgrade would have anything to do with it. I'll look into this next week.

Devtools' Storage tab lists

• 103 com.auth0.auth.... cookies (previously discussed in bug 1507454 also see below)
• a sessionid cookie (expires 2h after the login)
• a csrftoken one.

If the sessionid expires, is there supposed to be an automatic renewal or has the lifetime been shortened?

Regarding the many auth cookies:
Sheriff Narcis had ~15 auth cookies (oldest from Friday). My oldest is from Saturday. Does every treeherder tab initiate its own credential renewal? Sometimes cookies have expiration dates in a narrow time window, e.g. yesterday ~13:35 UTC, 19 cookies expired in 5 minutes. That Firefox still has the expired cookies is bug 691973.
That the issue can be reproduced in a new profile with one auth cookie shows this is unrelated.

Confirming that the logout happens after 2h when the sessionid cookie expires.

(FTR, the com.auth0 cookies count increases with every manual reload of a treeherder tab.)

These are all good questions Sebastian, thanks for the details. I'm looking into this today.

After some investigating there appears to be two issues:

1. We are receiving an access code from auth0 during login that includes an expires_in=7200 param (2 hours) instead of 24 hours. This might be an auth0 issue that our iam team/infosec might be able to help with. Bug 1611030 was filed and has more details.
2. The auto/silent renewing might not be working. After leaving myself logged in and looking at the console by chance, I noticed a Could not renew login:, { error: "timeout", errorDescription: "Timeout during authentication renew"} message. I'll continue looking into this part of the code tomorrow. At the very least, we should be showing the user is logged out when that error is caught.

Sheriffs, I'll be curious to see if you encounter the same error message next time you try to classify and get a message indicating you're not logged in (just check the console before refreshing).

Yes, the same error message can be found in the console here.

Update:

With the recent Django 3 upgrade, the default value of x-frame-options was changed from "sameorigin" to "deny" so this was causing the frequent timeouts/logging out issues (the auth0 silent renewal for the access token is set to run every 15 minutes - in an iframe - and resets if a new tab is opened I think).

For why the access token has an expiration of every 2 hours, It appears that we're using the implicit flow login for auth0 - since we're requesting id_token token in the UI instead of code, which returns an access token with an expiration of 2 hours instead of 24 hours.

I'd have to look more into what a change to the authorization flow for 24 hour access tokens would entail. So for now, the patch sets the x-frame-options back to sameorigin and fixes the logout mechanism so the UI updates when users are logged out due to a silent renewal failure.

Just made a minor tweak to the log out logic ^

I have other priorities to work on so closing out this bug. If anyone has other issues with the login/credentials - or would like us to explore a switch to the 24 hour access token - please file a new bug.