I suspect it's related with the changes from bug 1297519. The task[0] failed due to: --- ... 98% [237 libgtk-3-dev 14.4 kB/868 kB 2%] 6483 kB/s 0s 99% [Waiting for headers] 6483 kB/s 0s Err:149 http://archive.ubuntu.com/ubuntu xenial-security/main amd64 linux-libc-dev amd64 4.4.0-36.55 404 Not Found [IP: 91.189.88.161 80] 99% [Working] 6483 kB/s 0s Fetched 64.2 MB in 12s (4979 kB/s) E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/l/linux/linux-libc-dev_4.4.0-36.55_amd64.deb 404 Not Found [IP: 91.189.88.161 80] E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing? [0;31mCommand '['sudo', 'apt-get', '-o', 'Dpkg::Progress-Fancy=1', '--no-install-recommends', '-y', 'install', 'libgtk-3-dev', 'make', 'git']' returned non-zero exit status 100[0m --- [0]: https://tools.taskcluster.net/task-inspector/#ygq5FFJuRBihSvvFD02smQ/

Tracing back in https://public-artifacts.taskcluster.net/ygq5FFJuRBihSvvFD02smQ/4/public/logs/live_backing.log it seems that http://hg.mozilla.org/mozilla-central/file/tip/release/docker/firefox-snap/runme.sh#l45 is the culprit line. I don't think a simple "fix --missing" somewhere could solve this yet. Not sure how to fix this issue, need to investigate more on how this works.

I'm wondering if the docker image from rail's hub https://github.com/mozilla/releasetasks/pull/200/files#diff-51a0a141c4d0e9bb7104b16b6aa08372R29 is actually generated based on http://hg.mozilla.org/mozilla-central/file/tip/release/docker/firefox-snap/Dockerfile If that is true, maybe we can add some "fix --missing" somewhere.

I replicated the issue locally. With an "apt-get update --fix-missing" in the local Docker environment things worked just fine pass the hiccup. If my understanding is right as per bug 1297519 comments, it seems that rail used the http://hg.mozilla.org/mozilla-central/file/tip/release/docker/firefox-snap/Dockerfile to generate an image and upload it to his Dockerhub. Steps to do further: 1. Pushed a quick fix for the image, pending review 2. Rail needs to apply that and regenerate his image in Dockerhub that is used by snaps 3. Not sure if we need a follow-up build2 on 50.0b2 because of this. @rail: if I'm wrong, please discard the review changes.

Comment on attachment 8795219 [details] Bug 1305651 - fix missing Ubuntu archives in Firefox beta snaps generation. https://reviewboard.mozilla.org/r/81328/#review79918 Thanks for looking into this. I think thi is an issue with outdated package indexes when you run the script, not when you create the image. What you need is to run `apt-get update` before running `runme.sh` or before running `snapcraft` in the script.

"Edit & Create", added "apt-get update --fix-missing" resulted in a task passing that hiccup point - https://tools.taskcluster.net/task-inspector/#EgGpUaBsT1ewIrItorLt5g/0

(In reply to Mihai Tabara [:mtabara] from comment #6) > "Edit & Create", added "apt-get update --fix-missing" resulted in a task > passing that hiccup point - > https://tools.taskcluster.net/task-inspector/#EgGpUaBsT1ewIrItorLt5g/0 Task completed successfully. Pushed a patch to solve this for da future - https://github.com/mozilla/releasetasks/pull/203

I don't think it's worth leaving this bug open. We've already tested the change so hopefully no issues will come up in the next beta.

Seems like we're hitting the same issue yet again in 50.0b3. For some reason the fix I pushed in beta2 didn't work. Need to investigate more, see logs https://mozilla-release-logs.s3.amazonaws.com/mozilla-beta/firefox-50.0b3/build1/mozilla-beta_snap_generation-all-0wRjwRCRRyaTskLh6adr4A-0

If you look at the task definition, it looks like we forgot to deploy the changes. :)

(In reply to Rail Aliiev [:rail] from comment #10) > If you look at the task definition, it looks like we forgot to deploy the > changes. :) Ah, sorry about that, it's my fault, I was suppose to do that once I merged the PR. May I do the honors https://github.com/rail/build-ansible/blob/relpro/deploy_releasetasks.yml ?

(In reply to Mihai Tabara [:mtabara] from comment #11) > (In reply to Rail Aliiev [:rail] from comment #10) > > If you look at the task definition, it looks like we forgot to deploy the > > changes. :) > > Ah, sorry about that, it's my fault, I was suppose to do that once I merged > the PR. > May I do the honors > https://github.com/rail/build-ansible/blob/relpro/deploy_releasetasks.yml ? This is done now. I'll leave the bug open until the next beta confirms everything is fine.

50.0b4 https://tools.taskcluster.net/push-inspector/#/zrxQ87rIRR2KASmRlYFvjg/Rd48PZ8xRWShRaMsPGr8IA?_k=erkng2 snaps generation worked like a charm. We can now safely close this.