Raise error if active site is marked as out-of-date
Categories
(Firefox Build System :: Mach Core, enhancement)
Tracking
(firefox100 fixed)
Tracking | Status | |
---|---|---|
firefox100 | --- | fixed |
People
(Reporter: mhentges, Assigned: mhentges)
References
Details
Attachments
(3 files)
There's some cases in CI where the "system python" is used, and the build
site is activated multiple times.
Confusingly, the build
site is recreated each time, and this causes a Windows-specific error in CI (Access is denied: '$topsrcdir\\_virtualenvs\\build\\Scripts\\python.exe'
).
Let's guard against this in a platform-specific way, then resolve the issue.
Assignee | ||
Updated•3 years ago
|
Assignee | ||
Comment 1•3 years ago
|
||
Unless either the site's requirements or its on-disk virtualenv
changes, it shouldn't become out-of-date.
This will also catch situations where sites are incorrectly
being marked out-of-date due to sys.path
mismanagement.
Assignee | ||
Comment 2•3 years ago
|
||
So far, we've been using virtualenv
's activate_this.py
script.
However, unlike earlier expectations, it adds its sys.path
additions
to the front, not the back! This breaks our prioritization
requirements, such as:
- When using any package from the system environment, import all
possible packages from the system to avoid compatibility issues. - Use vendored packages instead of virtualenv-installed packages
wherever possible, because it more-closely matches developer
expectations ("why is this package vendored if it's not used?")
Define an activate_virtualenv()
function that replicates the logic
of activate_this.py
[1], except for three differences:
- Don't modify
sys.real_prefix
, since it's a non-standard change - Only add seen-with-
venv
-module paths to thesys.path
($prefix
,
$prefix/.../$site_packages_dir
) - don't do the paths in-between. - And, of course, append instead of prepend
sys.path
entries.
As an aside, this is one of the few remaining blockers from allowing
us to fully embrace venv
instead of virtualenv
- the last piece is
waiting on the fix for bug 1697833 to propagate.
[1]
https://github.com/pypa/virtualenv/blob/20.7.2/src/virtualenv/activation/python/activate_this.py
Depends on D140578
Assignee | ||
Comment 3•3 years ago
|
||
We want _pthfile_lines()
to consistently and accurately represent a
virtualenv's sys.path
modifications.
However, this becomes tricky when comparing expected pthfile_lines
//before// activating a virtualenv versus //afterwards//, especially
since Python implicitly adds some paths (such as the path to
site-packages
).
The current solution made this scrubbing only happen if we were
pip install
-ing into the site. Unfortunately, it //doesn't// work for
the "use system Python for the build
site", because:
- Pre-activation result:
site-packages
isn't added, because we aren't
using it - Post-activation result, implicitly-added
site-packages
isn't
scrubbed, because scrubbing only happened for
SitePackagesSource.VENV
Stepping back, the solution here is:
pthfile_lines
only represents Mach modifications //on top// of
implicit virtualenv behaviour: sincesite-packages
is always added
by default, it shouldn't be in our explicitpthfile_lines
.- The only time when implicit
sys.path
additions throws us off is when
SitePackagesSource.SYSTEM
: so, move the scrubbing to happen in that
case instead.
Finally refactor the "conditional deprioritization" comment to be more
useful and more accurate: we've implemented the "nontrivial complexity"
of purging site-packages, and the other piece about nothing being
pip install
-ed feels self-explanatory enough.
Depends on D140579
Updated•3 years ago
|
Updated•3 years ago
|
Updated•3 years ago
|
Updated•3 years ago
|
Updated•3 years ago
|
Comment 5•3 years ago
|
||
bugherder |
https://hg.mozilla.org/mozilla-central/rev/d0c262934162
https://hg.mozilla.org/mozilla-central/rev/1337faa66745
https://hg.mozilla.org/mozilla-central/rev/1883182d614f
Description
•