Closed Bug 1725708 Opened 3 years ago Closed 3 years ago

Include all possible vendored dependencies under "requirements.txt"

Categories

(Firefox Build System :: Mach Core, task)

task

Tracking

(firefox94 fixed)

RESOLVED FIXED
94 Branch
Tracking Status
firefox94 --- fixed

People

(Reporter: mhentges, Assigned: mhentges)

References

(Blocks 1 open bug)

Details

Attachments

(3 files, 1 obsolete file)

A fair amount of the packages in third_party/python/ have been copied in there manually, causing two downsides:

  • They aren't asserted to be compatible with each other
  • It's not obvious what some of the versions of the packages are

We should port them to use the centralized system.

Assignee: nobody → mhentges
Status: NEW → ASSIGNED

You can probably also remove the warning while you're here.

There's no usages of the pep487 module.

Depends on D123120

Note that, as part of adding this packages to the automated vendoring
system, some dependencies were automatically added - most notably,
dependencies of taskcluster that become visible with Python 3.6+.

Adds **/.git to the exclusions because:

  • .git is part of our .hgignore, but
  • .git is part of the aiohttp tar.gz file.

Since the file isn't needed for pip install-ing aiohttp,
and since we want ./mach vendor python to be a no-op when there's
no requirement changes, we exclude it.

Depends on D123121

Depends on D123122

Rather than deleting the expected target directory of each package
that's being vendored, clear the whole third_party/python directory
and re-populate it from scratch.

As part of this, there's an "exclusion" list for packages that can't
be vendored from PyPI.

This has some benefits:

  • It'll be harder to forget scraps of files and directories and leave
    them in third_party/python.
  • The exclusion list makes it more clear which packages are managed
    manually, and the friction it adds to the workflow will guide
    developers to use "requirements.in" instead.

The test_up_to_date_vendor test will verify that the vendor directory
is always clean.

Depends on D123123

Attachment #9237059 - Attachment is obsolete: true
Pushed by mhentges@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/11de3a4312bd Remove unused "pep487" vendored package r=ahal https://hg.mozilla.org/integration/autoland/rev/75336eb10063 Move all possible vendored deps to centralized system r=ahal https://hg.mozilla.org/integration/autoland/rev/90b9ff20b311 `./mach vendor python` should populate from a clean state r=ahal
Pushed by mhentges@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/746509f3a640 Remove unused "pep487" vendored package r=ahal https://hg.mozilla.org/integration/autoland/rev/06115f01c86f Move all possible vendored deps to centralized system r=ahal https://hg.mozilla.org/integration/autoland/rev/dd841f22c83e `./mach vendor python` should populate from a clean state r=ahal
Status: ASSIGNED → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
Target Milestone: --- → 94 Branch
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: