Closed Bug 1433033 Opened 7 years ago Closed 7 years ago

Various cleanups in the Debian-based docker images

Categories

(Firefox Build System :: Task Configuration, task)

Product:
Firefox Build System
Component:
Task Configuration
Type:
task
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED
Milestone:
mozilla60

People

(Reporter: glandium, Assigned: glandium)

References

Details

Attachments

(9 files)

Bug 1433033 - Remove a few packages from the toolchain-build image.
(deleted), text/x-review-board-request
dustin
: review+
Details
Bug 1433033 - Set Acquire::Check-Valid-Until in apt config file rather than pass it on the command line.
(deleted), text/x-review-board-request
dustin
: review+
Details
Bug 1433033 - Set apt defaults to avoid passing -q, -y and --no-install-recommends to it.
(deleted), text/x-review-board-request
dustin
: review+
Details
Bug 1433033 - Don't `apt-get clean` from Debian-based docker images.
(deleted), text/x-review-board-request
dustin
: review+
Details
Bug 1433033 - Create a base Debian docker image.
(deleted), text/x-review-board-request
dustin
: review+
Details
Bug 1433033 - Create a mercurial package for Debian stretch.
(deleted), text/x-review-board-request
dustin
: review+
Details
Bug 1433033 - Workarounds for Debian stretch.
(deleted), text/x-review-board-request
dustin
: review+
Details
Bug 1433033 - Derive the android-build image from a base Debian stretch image.
(deleted), text/x-review-board-request
dustin
: review+
Details
Bug 1433033 - Make APT retry when it fails to download files.
(deleted), text/x-review-board-request
dustin
: review+
Details
No description provided.
Assignee: nobody → mh+mozilla
Depends on: 1399679
Comment on attachment 8945429 [details] Bug 1433033 - Derive the android-build image from a base Debian stretch image. https://reviewboard.mozilla.org/r/215610/#review221408
Attachment #8945429 - Flags: review?(dustin) → review+
Comment on attachment 8945428 [details] Bug 1433033 - Workarounds for Debian stretch. https://reviewboard.mozilla.org/r/215608/#review221410
Attachment #8945428 - Flags: review?(dustin) → review+
Attachment #8945427 - Flags: review?(dustin) → review+
Comment on attachment 8945427 [details] Bug 1433033 - Create a mercurial package for Debian stretch. https://reviewboard.mozilla.org/r/215606/#review221406 ::: taskcluster/ci/packages/kind.yml:99 (Diff revision 1) > + sha256: dc2f72caccd6b760226753d48c2f4e8889fe176a6b23ef50775caac55ce28b85 > + pre-build-command: >- > + cp -r contrib/debian debian && > + sed -i -e "s/__VERSION__/$(awk -F\" '$2 {print $2}' mercurial/__version__.py)-1.deb9moz1/" \ > + -e "s/__DATE__/$(date --rfc-2822)/" \ > + -e "s/__CODENAME__/wheezy/" debian/changelog stretch?
Comment on attachment 8945426 [details] Bug 1433033 - Create a base Debian docker image. https://reviewboard.mozilla.org/r/215604/#review221404 This is great!
Attachment #8945426 - Flags: review?(dustin) → review+
Comment on attachment 8945425 [details] Bug 1433033 - Don't `apt-get clean` from Debian-based docker images. https://reviewboard.mozilla.org/r/215602/#review221402
Attachment #8945425 - Flags: review?(dustin) → review+
Comment on attachment 8945424 [details] Bug 1433033 - Set apt defaults to avoid passing -q, -y and --no-install-recommends to it. https://reviewboard.mozilla.org/r/215600/#review221400
Attachment #8945424 - Flags: review?(dustin) → review+
Comment on attachment 8945423 [details] Bug 1433033 - Set Acquire::Check-Valid-Until in apt config file rather than pass it on the command line. https://reviewboard.mozilla.org/r/215598/#review221398
Attachment #8945423 - Flags: review?(dustin) → review+
Comment on attachment 8945422 [details] Bug 1433033 - Remove a few packages from the toolchain-build image. https://reviewboard.mozilla.org/r/215596/#review221396
Attachment #8945422 - Flags: review?(dustin) → review+
Comment on attachment 8945428 [details] Bug 1433033 - Workarounds for Debian stretch. https://reviewboard.mozilla.org/r/215608/#review221480 ::: taskcluster/docker/debian-base/cloud-mirror-workaround.sh:7 (Diff revision 1) > + > +# The apt in Debian stretch mishandles the HTTP redirects from queue.taskcluster.net to > +# cloud-mirror.taskcluster.net, and unescapes the url. This apt method wrapper > +# re-escapes the url. > + > +sed -u '/^URI: https:\/\/cloud-mirror.taskcluster.net\/v1\/redirect\//{s,/,%2F,8g;s,:,%3A,2g}' | /usr/lib/apt/methods/https There are still problems with this hack :(
Comment on attachment 8945428 [details] Bug 1433033 - Workarounds for Debian stretch. (In reply to Mike Hommey [:glandium] from comment #17) > There are still problems with this hack :( Okay, this new version works, but it does so quite differently, so I'd rather get another review for it. Please note that the sed is effectively a no-op on Debian wheezy (because there, the https method does return the escaped url unchanged), so this only would make stretch-based docker images hit the us-west-2 region. This would also effeectively stop doing anything when https://github.com/taskcluster/cloud-mirror/issues/39 is fixed.
Attachment #8945428 - Flags: review+ → review?(dustin)
Comment on attachment 8945428 [details] Bug 1433033 - Workarounds for Debian stretch. https://reviewboard.mozilla.org/r/215608/#review221758 ::: taskcluster/docker/debian-base/cloud-mirror-workaround.sh:8 (Diff revision 2) > +# The apt in Debian stretch mishandles the HTTP redirects from queue.taskcluster.net to > +# cloud-mirror.taskcluster.net, and unescapes the url. This apt method wrapper > +# strips redirections and sends directly to the S3 bucket. This has the downside of > +# always hitting us-west-2 independently of the zone this runs on, but this is only > +# used when creating docker images, so shouldn't generate huge cross-AWS-zone S3 > +# transfers. ++ good fix.
Attachment #8945428 - Flags: review?(dustin) → review+
Comment on attachment 8945682 [details] Bug 1433033 - Make APT retry when it fails to download files. https://reviewboard.mozilla.org/r/215816/#review221760
Attachment #8945682 - Flags: review?(dustin) → review+
Pushed by mh@glandium.org: https://hg.mozilla.org/integration/autoland/rev/7ec508c2094f Remove a few packages from the toolchain-build image. r=dustin https://hg.mozilla.org/integration/autoland/rev/18bbec0d7d5e Set Acquire::Check-Valid-Until in apt config file rather than pass it on the command line. r=dustin https://hg.mozilla.org/integration/autoland/rev/06acfb703485 Set apt defaults to avoid passing -q, -y and --no-install-recommends to it. r=dustin https://hg.mozilla.org/integration/autoland/rev/f3b938ae517b Don't `apt-get clean` from Debian-based docker images. r=dustin https://hg.mozilla.org/integration/autoland/rev/ff9ce55c9291 Create a base Debian docker image. r=dustin https://hg.mozilla.org/integration/autoland/rev/e1e8d1028b17 Create a mercurial package for Debian stretch. r=dustin https://hg.mozilla.org/integration/autoland/rev/3bebaac29a15 Workarounds for Debian stretch. r=dustin https://hg.mozilla.org/integration/autoland/rev/90650d0682f1 Derive the android-build image from a base Debian stretch image. r=dustin https://hg.mozilla.org/integration/autoland/rev/b57d3491c10c Make APT retry when it fails to download files. r=dustin
Pushed by mozilla@hocat.ca: https://hg.mozilla.org/comm-central/rev/caa15de76ecf Port Bug 1433033: Create a base Debian docker image; r=me
Blocks: 1436283
Product: TaskCluster → Firefox Build System
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: