I wanted to test out bug 1615932 but the latest firefox-snap docker image is busted: it cannot start snapcraft[1]:

[...]
Reading package lists... 99%
Reading package lists... Done
+ snapcraft
/snap/bin/snapcraft: 3: exec: /snap/snapcraft/current/usr/bin/python3: not found

I started to investigate the problem and noticed I cannot even run python on this docker image. This is not an infra issue, I am able to reproduce the problem locally. Neither strace, nor ldd, not readelf enabled me to understand why the python reported a not found error even though the binary does exist and so do its dynamic libraries.

That said, I found a fix! Bug 1591751 changed the Dockerfile to be as close as possible as the Dockerfile define in snapcraft. It turns out a new patch has modified this file since then. I backported it and it somehow worked.

[1] https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=289198465&repo=try&lineNumber=1946
[2] https://github.com/sergiusens/snapcraft/blob/194ae904ec95031c76800920e72ab29863ca4d2f/docker/stable.Dockerfile
[3] https://github.com/snapcore/snapcraft/pull/2883

This bug will impact all branches once taskgraph decides [1] needs to be rebuilt.

[1] https://firefox-ci-tc.services.mozilla.com/tasks/eu4FX21mTPy-7wxz94cpvg

https://hg.mozilla.org/mozilla-central/rev/bae04ecb4e26

Comment on attachment 9127094 [details]
Bug 1616048 - Fix firefox-snap docker image r=rail

Beta/Release Uplift Approval Request

• User impact if declined: See comment 1. Snap repackages won't be able to be built at some point. This will prevent users from getting new Snap updates and will cause unexpected breakages in release promotion.
• Is this code covered by automated tests?: No
• Has the fix been verified in Nightly?: No
• Needs manual test from QE?: No
• If yes, steps to reproduce: N/A
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): This change is a backport of what Canonical has been doing in production (see comment 0).
• String changes made/needed: None

ESR Uplift Approval Request

• If this is not a sec:{high,crit} bug, please state case for ESR consideration: Same as User impact defined above
• User impact if declined: Same as above
• Fix Landed on Version:
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): Same as above
• String or UUID changes made by this patch: None

Comment on attachment 9127094 [details]
Bug 1616048 - Fix firefox-snap docker image r=rail

Avoids hitting issues with Snap repacks in the future. Approved for all the branches.

https://hg.mozilla.org/releases/mozilla-beta/rev/f361279177ed00ffd80a0203d5b42a96a6868c1a
https://hg.mozilla.org/releases/mozilla-release/rev/35bba5f9548540d5b0a38ec802ae6fe164ff3d74

Comment on attachment 9127094 [details]
Bug 1616048 - Fix firefox-snap docker image r=rail

Johan says that ESR68 isn't affected after all.

Yes, bug 1591751 has never made esr68. Therefore esr68 is already picking whatever Canonical publishes. In other words, esr68 will get the fix automatically.

I'm sorry, I should have remembered that before asking for all approvals! Thanks Ryan for confirming this with me 😃