https://tools.taskcluster.net/groups/B5SChkanQf2ahu0Kr1POgQ/tasks/D9L1VHbUTLC_eNmaj-4bkA/runs/0/logs/public%2Flogs%2Flive_backing.log: > 2018-08-06 12:15:58,061 - bouncerscript.script - INFO - Sanity check versions and aliases before updating ... > 2018-08-06 12:15:58,061 - scriptworker.client - ERROR - Failed to run async_main > Traceback (most recent call last): > File "/builds/scriptworker/lib/python3.6/site-packages/scriptworker/client.py", line 203, in _handle_asyncio_loop > await async_main(context) > File "/builds/scriptworker/lib/python3.6/site-packages/bouncerscript/script.py", line 99, in async_main > await action_map[context.action](context) > File "/builds/scriptworker/lib/python3.6/site-packages/bouncerscript/script.py", line 70, in bouncer_aliases > check_product_names_match_aliases(context) > File "/builds/scriptworker/lib/python3.6/site-packages/bouncerscript/task.py", line 88, in check_product_names_match_aliases > raise TaskVerificationError("Unrecognized alias:{}".format(alias)) > scriptworker.exceptions.TaskVerificationError: Unrecognized alias:thunderbird-esr-next-latest This entry doesn't exist on the bouncer side. It should be "thunderbird-latest" instead. Moreover there is no SSL entry for thunderbird release referenced in bouncer. We may probably want to either create it, or just remove the in-tree definition.

In the meantime, I manually changed thunderbird-latest alias from "Thunderbird-52.9.1" to "Thunderbird-60.0"

While shipping TB60, bouncerscript sanity caught some error that `thunderbird-esr-next-latest` didn't exist on bouncer. Which was true. We don't have that neither in bouncerscript regexes, nor in bouncer. I took a moment and double-checked we're up-to-date with entries in bouncer aliases. Ran a quick script to double-check what we currently have under https://github.com/mozilla-releng/bouncerscript/blob/master/bouncerscript/constants.py#L1 against official bouncer entries and I got: ``` (Pdb) pp official_set.difference(our_set) {'firefox-esr52-latest-ssl', 'firefox-esr52-latest'} (Pdb) pp our_set.difference(official_set) {'thunderbird-latest-ssl', 'thunderbird-next-latest', 'thunderbird-next-latest-ssl'} ``` So, to begin with 1. i) We don't have the esr52 aliases included here. That's expected since that's not subject to bouncerscript these days. ii) We have three additional aliases defined in our regexes that are nowhere to be found in bouncer, so most likely never used by other pieces of automation. @tomprince: are we okay with removing these three aliases below from our sanity validation check? CCing @JohanLorenzo ``` {'thunderbird-latest-ssl', 'thunderbird-next-latest', 'thunderbird-next-latest-ssl'} ``` 2. Sounds like we need to fix the in-tree aliases for thunderbird to correctly submit the payload containing the `thunderbird-latest` instead.

:sancus, does thunderbird use any bouncer aliases, or is there any desire to? Firefox[1] uses URLs like https://download.mozilla.org/?product=firefox-latest-ssl&os=linux64&lang=en-US that don't contain a version, and get updated to point to the most recent version as part of shipping the release. I set this up as part of the taskcluster migration, but as indicated above, there was some issues with how this is setup. Either we can fix it, or just remove the code that updates the aliases. [1] https://www.mozilla.org/en-US/firefox/download/

We don't use these aliases right now, the website is aware of what version is the latest. I don't think these aliases would remove that need, since eg we still need to display the correct version numbers on release notes and elsewhere. Maybe there is some potential future use for them but I don't think we need them right now. At least not on the website.

:rjl, given the above, we should remove the bouncer-alias tasks from comm-*.

Removed release-bouncer-aliases dependencies and removed the task configuration.

Comment on attachment 9012368 [details] Bug 1481198 - Remove bouncer-alias tasks from comm-*. r?tomprince Tom Prince [:tomprince] has approved the revision.

These are referenced in http://ftp.mozilla.org/pub/thunderbird/releases/latest/README.txt so if the decision is taken to proceed with deletion rather than fixing, the README should also be updated.

Please reconsider removing this functionality. Many of us (as USERS) use this ( as systems administrators, etc?) to try to automate downloading of NEW releases. We've been led to believe that we can rely on these "bouncer"(?) URLs to point us to stable release versions. e.g. using a lookup like: app_url="${moz_proto}://${moz_site}/?product=${app_name}&os=${app_os}&lang=${app_lang}" app_fname="$(curl ${curl_verbose} --silent --head "${app_url}" | sed -ne '/^Location:/{s/^.*\///;s/\r$//;p}')" to test whether i already have the latest version downloaded in a staging area. Trying to figure out how to get to the actual download locations (since ftp://ftp.mozilla.org is no longer around) going through all the "helpful" (not) automated web interfaces is really not friendly at all to enterprise management, so if you remove this mechanism, please replace it with some well-defined language on how we can get directly to the download URLs for current packages. (e.g. i accidentally grabbed the Linux32 TB 60.2.1 via Web clicky on the penguin icon, and that sucked for a bit while i was trying to figure out why libraries were missing :-( ) PROGRAMMATIC FOR THE WIN! thanks, --stephen

FYI, this thread is about this change: https://mail.mozilla.org/pipermail/enterprise/2018-October/000327.html

(In reply to sdowdy from comment #10) > Please reconsider removing this functionality. Many of us (as USERS) use > this ( as systems administrators, etc?) to try to automate downloading of > NEW releases. We've been led to believe that we can rely on these > "bouncer"(?) URLs to point us to stable release versions. FYI, the source of truth for Thunderbird versions(used by all thunderbird.net websites as well) is https://product-details.mozilla.org/1.0/thunderbird_versions.json -- this is automatically updated when a new version is marked shipped by release engineering. We may yet keep the bouncer aliases though, that has certainly not been decided or this bug would be closed. :)

> FYI, the source of truth for Thunderbird versions(used by all > thunderbird.net websites as well) is > https://product-details.mozilla.org/1.0/thunderbird_versions.json Thanks, that's handy to know. My concern with using the json file is mapping from the version number to a download URL. If the URLs follow a standard format that is fairly easy to compute and doesn't change (or rarely changes), that might be an acceptable solution for my case.

(In reply to Paul Sparks from comment #13) > If the URLs follow a standard format that is fairly easy to > compute and doesn't change (or rarely changes), that might be an acceptable > solution for my case. They follow the format https://download.mozilla.org/?product=thunderbird-60.2.1-SSL&os=win&lang=en-US - while I can't guarantee that this will never, ever change, it hasn't changed in many years and any change would likely be accompanied by a rewrite for the old URLs anyway.

comm-esr60 is really a regular release branch, so it should be referring to thunderbird-latest and thunderbird-latest-ssl. Note that the comm-beta configuration in this file does work as expected.

Attachment 9020435 [details] is the fix. Plan is to get it into esr60 in time for the next release.

Comment on attachment 9020435 [details] Bug 1481198 - update bouncer-aliases for comm-esr60 to correct mapping. r?darktrojan [ESR Uplift Approval Request] If this is not a sec:{high,crit} bug, please state case for ESR consideration: This fixes the thunderbird-latest bouncer at download.mozilla.org User impact if declined: The bouncer continues to point to an old version for ESR releases. Beta works, but only because thunderbird-latest-beta points to a valid entry. Once that merges to esr60 it will be broken for that release. Fix Landed on Version: not yet in c-c Risk to taking this patch: Low Why is the change risky/not risky? (and alternatives if risky): Change will only affect the release process if it fails. The bouncer is broken now for releases, it won't be any more broken if this fails. String or UUID changes made by this patch: [Beta/Release Uplift Approval Request] Feature/Bug causing the regression: None User impact if declined: Is this code covered by automated tests?: Yes Has the fix been verified in Nightly?: Yes Needs manual test from QE?: Yes If yes, steps to reproduce: List of other uplifts needed: None Risk to taking this patch: Low Why is the change risky/not risky? (and alternatives if risky): String changes made/needed:

Comment on attachment 9020435 [details] Bug 1481198 - update bouncer-aliases for comm-esr60 to correct mapping. r?darktrojan [Approval Request Comment] Regression caused by (bug #): User impact if declined: Testing completed (on c-c, etc.): Risk to taking this patch (and alternatives if risky): [Approval Request Comment] Regression caused by (bug #): User impact if declined: Testing completed (on c-c, etc.): Risk to taking this patch (and alternatives if risky):

Pushed by mozilla@jorgk.com: https://hg.mozilla.org/comm-central/rev/50073ff4e729 update bouncer-aliases for comm-esr60 to correct mapping. r=darktrojan

TB 60.3 ESR: https://hg.mozilla.org/releases/comm-esr60/rev/5d579df31f08d740f8372723a8b9252af3ed76e3

Beta (TB 64 beta 2): https://hg.mozilla.org/releases/comm-beta/rev/3e08c32a7ddb2652481f584f82c275247531cb35