Open
Bug 1768001
Opened 3 years ago
Updated 3 years ago
Improve lockfile generation performance: remove setup.py dependencies
Categories
(Firefox Build System :: Mach Core, enhancement, P3)
Firefox Build System
Mach Core
Tracking
(Not tracked)
NEW
People
(Reporter: mhentges, Unassigned)
References
(Depends on 1 open bug)
Details
Each lockfile takes ~15s to create, but most of this time is spinning up a Python interpreter to invoke setup.py
for a few of our packages:
third_party/python/gyp
(third party)third_party/python/dlmanager
(third party)testing/web-platform/tests/tools/wptrunner
(first party)testing/web-platform/tests/tools/wptserve
(first party)testing/web-platform/tests/tools/third_party/mod_pywebsocket
(third party)
We should do the following:
- Convert our first-party packages to use
pyproject.toml
instead ofsetup.py
: this allows them to still be trivially modified, yet far cheaper forpoetry
to check dependency information/etc (reading TOML is cheaper than interpreting a Python file, how strange is that? 🤔) - Process dist info for our third party packages
- As part of this, we can tweak our Poetry integration (
python_lockfile.py
) to no longer copysetup.py
packages before processing them.
- As part of this, we can tweak our Poetry integration (
- Add a test to
python/mach/
that verifies that no site has asetup.py
package with outdated dist information.
By allowing Poetry to run without having to invoke any setup.py
files, we should be able to drop per-lockfile generation time to ~1s each, which is very decent.
Updated•3 years ago
|
Severity: -- → S3
Priority: -- → P3
You need to log in
before you can comment on or make changes to this bug.
Description
•