everyone upgrading from older thunderbird should go through version 68.5+ to ensure we migrate to nss key4.db and cleanup key3.db

Works for me.

For clarification, the migration and cleanup process happens starting with version 68.5.0? What about for beta/nightly? A similar watershed should get set up on those channels as well.

Right now we don't have a 68.5.1 on the calendar, so let's plan on 68.5.0 being the watershed version. The 68.6.0 release then should probably mention the watershed.

(In reply to Rob Lemley [:rjl] from comment #1)

For clarification, the migration and cleanup process happens starting with version 68.5.0? What about for beta/nightly? A similar watershed should get set up on those channels as well.

It's more complicated.

The migration started happening with 58 beta, or 60.0 release.

The cleanup only started to be done with 68.5 (because earlier we didn't have a way to avoid dataloss).

That means, everyone coming from <= 57.x should to through a wastershed, that's at least version 68.5.0, but any later 68.x will be fine, too.

For the cleanup, the watershed will be partially optimistic, because it won't be automatic for users who have set a master password. If they come from <= 57, start >= 68.x, don't login, but go straight on to release 78, those won't get the migration - because 72 was the last version that contained the migration code.

However, for anyone who has NOT set a master password, migration will happen as soon as starting 68.5+ once. Cleaning up will require that the 68.5+ execution obtains at least one password from storage, which will happen if e.g. automatic mail check is enabled.

In other words, the watershed won't be perfect, but it will help most users.

For beta/nightly regarding migration, everyone following those releases has already gotten the migration long ago. Regarding cleanup, only the 60.x and 68.x TB release branch used a patch to prevent the cleanup (the cleanup is by default in mozilla-central code). So everyone using any nightly >= 69 or <= 72 has already gotten the cleanup. I conclude we don't need a watershed for nightly/beta.

Okay, so if we can wait I would prefer to. I'll be setting up a Balrog rule starting with Thunderbird 68.10 that initially prevents auto-updating from 68 to 78. So that can effectively be the watershed rule.
FWIW, this is how 60 -> 68 is/was done. There's a watershed on 60.9.1, though it started out on 60.8 to coincide with the release of 68.0.

Lets reuse rule 954 for this.

Changes to make:

(Rule 954)
Version to match: <78.0 (effectively will be 60.9.1 <= x < 78.0 because of watershed at 60.9.1)
Version to send: [Latest 68 release]
Fallback version: [Set to something appropriate]
Rate: [Set to whatever is appropriate today]

(Rule 906)
Version to match: >=78.0

On comm-esr68 ONLY this will require a change to comm/taskcluster/ci/release-balrog-scheduling/kind.yml at line 36. (Change the 906 to 954)

This will mean that the watershed migrates from 68.9.0 to 68.10, 68.11, and will end up at 68.12 eventually.

Rule changes SGTM.

Maybe we should also have some KB notes for people providing support, and for users who get in a jam - even if it is short? Or do we already?

[Approval Request Comment]
Patch is only for comm-esr68.

I have the changes described above staged on release-localtest and release-cdntest.
The patch in comment 6 will need to be applied to comm-esr68 once the rule changes are made on the actual release channel. I'd like to do this for 68.10.

Do we have other watershed moments? Or is this the first in the life of the project. I have always been informed that the code to update has been left into subsequent versions. Has that changed?

(In reply to Matt from comment #9)

Do we have other watershed moments? Or is this the first in the life of the project. I have always been informed that the code to update has been left into subsequent versions. Has that changed?

There have been several over the years for various reasons, mostly dropping hardware support or OS support. This is what's in AUS.

38.5.0 - SHA1 certificate stuff on AUS servers (if anyone is running anything less than this version they cannot use AUS)
48.5.0 - require gtk > 3.4, drop macos < 10.9, require sse2
52.9.1 - last version to support win xp/vista
60.9.1 - last version to support bz2 formatted MAR files

68.10

Thunderbird 68.10.0:
https://hg.mozilla.org/releases/comm-esr68/rev/2fa6240eaa17

The AUS rule changes were made with the 68.10.0 release.

Did you remove that reminder for a reason Wayne? ( is there an issue with using NI as a constant reminder) I still intend to follow that up but life has been winning of late.

(In reply to Matt from comment #13)

Did you remove that reminder for a reason Wayne? ( is there an issue with using NI as a constant reminder) I still intend to follow that up but life has been winning of late.

Just a bit of bugzilla cleanup - I was thinking it was no longer needed, but I see this is about comment 8. Just NI yourself to add it back.