Closed Bug 1827319 Opened 2 years ago Closed 2 years ago

Perma bouncer-check(Firefox and Devedition) FAIL: https://download.mozilla.org/?product=Firefox/Devedition-113.0b1-stub&os=win&lang=de, status: 404

Categories

(Release Engineering :: Release Automation: Uploading, defect, P1)

Tracking

(firefox-esr102 unaffected, firefox112 unaffected, firefox113blocking fixed, firefox114 fixed)

RESOLVED FIXED
Tracking Status
firefox-esr102 --- unaffected
firefox112 --- unaffected
firefox113 blocking fixed
firefox114 --- fixed

People

(Reporter: intermittent-bug-filer, Assigned: jlorenzo)

References

(Regression)

Details

(Keywords: intermittent-failure, regression)

Attachments

(2 files)

Filed by: ctuns [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=411911305&repo=mozilla-beta
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/cV6trskRTx-dWW_Q5ZrZyw/runs/0/artifacts/public/logs/live_backing.log


[task 2023-04-11T00:53:16.483Z] 00:53:16     INFO - Checking https://download.mozilla.org/?product=Firefox-beta-stub&os=win&lang=en-US
[task 2023-04-11T00:53:16.485Z] 00:53:16     INFO - Checking https://download.mozilla.org/?product=Firefox-beta-stub&os=win&lang=de
[task 2023-04-11T00:53:16.488Z] 00:53:16     INFO - Checking https://download.mozilla.org/?product=Firefox-beta-stub&os=win&lang=it
[task 2023-04-11T00:53:16.489Z] 00:53:16     INFO - Checking https://download.mozilla.org/?product=Firefox-beta-stub&os=win&lang=zh-TW
[task 2023-04-11T00:53:16.493Z] 00:53:16     INFO - Checking https://download.mozilla.org/?product=Firefox-beta-stub&os=win64&lang=en-US
[task 2023-04-11T00:53:16.497Z] 00:53:16     INFO - Checking https://download.mozilla.org/?product=Firefox-beta-stub&os=win64&lang=de
[task 2023-04-11T00:53:16.499Z] 00:53:16    ERROR - FAIL: https://download.mozilla.org/?product=Firefox-113.0b1-stub&os=win&lang=de, status: 404
[task 2023-04-11T00:53:16.501Z] 00:53:16     INFO - Checking https://download.mozilla.org/?product=Firefox-beta-stub&os=win64&lang=it
[task 2023-04-11T00:53:16.502Z] 00:53:16     INFO - Checking https://download.mozilla.org/?product=Firefox-beta-stub&os=win64&lang=zh-TW
[task 2023-04-11T00:53:16.505Z] 00:53:16    ERROR - FAIL: https://download.mozilla.org/?product=Firefox-113.0b1-stub&os=win&lang=zh-TW, status: 404
[task 2023-04-11T00:53:16.509Z] 00:53:16    ERROR - FAIL: https://download.mozilla.org/?product=Firefox-113.0b1-stub&os=win&lang=it, status: 404
[task 2023-04-11T00:53:16.516Z] 00:53:16     INFO - Checking https://download.mozilla.org/?product=Firefox-beta-stub&os=win64-aarch64&lang=en-US
[task 2023-04-11T00:53:16.519Z] 00:53:16     INFO - Checking https://download.mozilla.org/?product=Firefox-beta-stub&os=win64-aarch64&lang=de
[task 2023-04-11T00:53:16.521Z] 00:53:16    ERROR - FAIL: https://download.mozilla.org/?product=Firefox-113.0b1-stub&os=win64&lang=it, status: 404
[task 2023-04-11T00:53:16.523Z] 00:53:16    ERROR - FAIL: https://download.mozilla.org/?product=Firefox-113.0b1-stub&os=win64&lang=de, status: 404
[task 2023-04-11T00:53:16.528Z] 00:53:16    ERROR - FAIL: https://download.mozilla.org/?product=Firefox-113.0b1-stub&os=win64&lang=zh-TW, status: 404
[task 2023-04-11T00:53:16.531Z] 00:53:16    ERROR - FAIL: https://download.mozilla.org/?product=Firefox-113.0b1-stub&os=win64-aarch64&lang=de, status: 404
[task 2023-04-11T00:53:16.534Z] 00:53:16     INFO - Checking https://download.mozilla.org/?product=Firefox-beta-stub&os=win64-aarch64&lang=it
[task 2023-04-11T00:53:16.535Z] 00:53:16    ERROR - FAIL: https://download.mozilla.org/?product=Firefox-113.0b1-stub&os=win64-aarch64&lang=it, status: 404
[task 2023-04-11T00:53:16.536Z] 00:53:16     INFO - Checking https://download.mozilla.org/?product=Firefox-beta-stub&os=win64-aarch64&lang=zh-TW
[task 2023-04-11T00:53:16.538Z] 00:53:16     INFO - Checking https://download.mozilla.org/?product=Firefox-113.0b1-langpack-SSL&os=linux&lang=en-US
[task 2023-04-11T00:53:16.539Z] 00:53:16    ERROR - FAIL: https://download.mozilla.org/?product=Firefox-113.0b1-stub&os=win64-aarch64&lang=zh-TW, status: 404
[task 2023-04-11T00:53:16.540Z] 00:53:16     INFO - Checking https://download.mozilla.org/?product=Firefox-113.0b1-langpack-SSL&os=linux&lang=de
[task 2023-04-11T00:53:16.543Z] 00:53:16     INFO - Checking https://download.mozilla.org/?product=Firefox-113.0b1-langpack-SSL&os=linux&lang=it
[task 2023-04-11T00:53:16.546Z] 00:53:16     INFO - Checking https://download.mozilla.org/?product=Firefox-113.0b1-langpack-SSL&os=linux&lang=zh-TW
[task 2023-04-11T00:53:16.548Z] 00:53:16     INFO - Checking https://download.mozilla.org/?product=Firefox-113.0b1-langpack-SSL&os=linux64&lang=en-US
[task 2023-04-11T00:53:16.549Z] 00:53:16     INFO - Checking https://download.mozilla.org/?product=Firefox-113.0b1-langpack-SSL&os=linux64&lang=de
[task 2023-04-11T00:53:16.551Z] 00:53:16     INFO - Checking https://download.mozilla.org/?product=Firefox-113.0b1-langpack-SSL&os=linux64&lang=it
[task 2023-04-11T00:53:16.553Z] 00:53:16     INFO - Checking https://download.mozilla.org/?product=Firefox-113.0b1-langpack-SSL&os=linux64&lang=zh-TW
[task 2023-04-11T00:53:16.567Z] 00:53:16     INFO - Checking https://download.mozilla.org/?product=Firefox-113.0b1-langpack-SSL&os=osx&lang=en-US

I still haven't gotten to the bottom of it. But, this is what I found.

The Firefox Installer.exe file in the pub/firefox/releases/113.0b1/win32/ l10n paths for 113.0b1 is not there.

See for example 113.0b1 win32 de vs. 112.0b9 win32 de

The file has been missing since the candidate build. That is to say, the file is not in the candidate directories for the 113.0b1 release and that is why the firefox-push-to-release task did not copy it into the release folder.

See for example 113.0b1 candidates win32 de vs. 113.0b1 candidates win32 de

The file is coming from the attribution kinds. See taskcluster/gecko_taskgraph/manifests/firefox_candidates.yml#355-366

The patch on revision D170246 touches taskcluster/gecko_taskgraph/manifests/firefox_candidates.yml and switches the upstream task for Firefox Installer.exe from repackage-signing to attribution and attribution-l10n (the en-US folders did end up with a Firefox Installer.exe file. See 113.0b1 win32 en-US)

This bug might be a regression related this stub attribution patch from Bug 1814727

Severity: S4 → S2
Keywords: regression
Priority: P5 → P1
Flags: needinfo?(bhearsum)
Assignee: nobody → jlorenzo
Status: NEW → ASSIGNED

Thank you very much :gabriel for your thorough analysis in comment 1! This made the bug much easier to narrow down 😃

Pushed by jlorenzo@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/10242bba12ac Beetmove localized stub installers again after the attribution split r=jcristau DONTBUILD
Component: Release Automation: Other → Release Automation: Uploading
QA Contact: gbrown → hneiva
Regressed by: 1814727

Comment on attachment 9327890 [details]
Bug 1827319 - Beetmove localized stub installers again after the attribution split r=jcristau a=bustage

Beta/Release Uplift Approval Request

  • User impact if declined: Can't shipped localized Windows Stub installers anymore.
  • Is this code covered by automated tests?: Yes
  • Has the fix been verified in Nightly?: No
  • Needs manual test from QE?: No
  • If yes, steps to reproduce:
  • List of other uplifts needed: None
  • Risk to taking this patch: Low
  • Why is the change risky/not risky? (and alternatives if risky): Not risky. The bug was identified by our end-to-end tests in the release pipeline. We'll be able to iterate if the issue remains.
  • String changes made/needed: None
  • Is Android affected?: No
Attachment #9327890 - Flags: approval-mozilla-beta?
Attachment #9327889 - Flags: approval-mozilla-beta?
Attachment #9327889 - Flags: approval-mozilla-beta?
Attachment #9327890 - Flags: approval-mozilla-beta? → approval-mozilla-beta+
Status: ASSIGNED → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
Flags: needinfo?(bhearsum)
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: