Error running `./mach vendor python`
Categories
(Firefox Build System :: Mach Core, defect, P2)
Tracking
(firefox-esr102 unaffected, firefox106 wontfix, firefox107 wontfix, firefox108 wontfix, firefox109 wontfix, firefox113 fixed)
People
(Reporter: ahal, Assigned: ahochheiden)
References
(Depends on 1 open bug, Blocks 1 open bug, Regression)
Details
(Keywords: regression)
Attachments
(7 files, 1 obsolete file)
(deleted),
text/x-phabricator-request
|
Details | |
(deleted),
text/x-phabricator-request
|
Details | |
(deleted),
text/x-phabricator-request
|
Details | |
(deleted),
text/x-phabricator-request
|
Details | |
(deleted),
text/x-phabricator-request
|
Details | |
(deleted),
text/x-phabricator-request
|
Details | |
(deleted),
text/x-phabricator-request
|
Details |
When I run ./mach vendor python
(even on a clean tree) I get:
$ ./mach vendor python
created virtual environment CPython3.10.5.final.0-64 in 17ms
creator CPython3Posix(dest=/home/ahal/dev/mozilla-unified/obj-x86_64-pc-linux-gnu/_virtualenvs/vendor, clear=False, no_vcs_ignore=False, global=False)
activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator
Collecting poetry==1.2.0a2
Using cached poetry-1.2.0a2-py3-none-any.whl (190 kB)
<snip>
Collecting jsmin==2.1.0
Using cached jsmin-2.1.0.tar.gz (9.6 kB)
Collecting json-e==2.7.0
Using cached json-e-2.7.0.tar.gz (17 kB)
Collecting jsonschema==3.2.0
Using cached jsonschema-3.2.0-py2.py3-none-any.whl (56 kB)
ERROR: Could not find a version that satisfies the requirement looseversion==1.0.1markupsafe==1.1.1 (from versions: 1.0.0, 1.0.1)
ERROR: No matching distribution found for looseversion==1.0.1markupsafe==1.1.1
Error running mach:
['vendor', 'python']
The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.
You can invoke |./mach busted| to check if this issue is already on file. If it
isn't, please use |./mach busted file vendor| to report it. If |./mach busted| is
misbehaving, you can also inspect the dependencies of bug 1543241.
If filing a bug, please include the full output of mach, including this error
message.
The details of the failure are as follows:
subprocess.CalledProcessError: Command '['/home/ahal/dev/mozilla-unified/obj-x86_64-pc-linux-gnu/_virtualenvs/vendor/bin/python', '-m', 'pip', 'download', '-r', '/tmp/tmpc8w78ai_/requirements.no-markers.txt', '--no-deps', '--dest', '/tmp/tmpi1j4uks3', '--abi', 'none', '--platform', 'any']' returned non-zero exit status 1.
File "/home/ahal/dev/mozilla-unified/python/mozbuild/mozbuild/vendor/mach_commands.py", line 229, in vendor_python
vendor_command.vendor(keep_extra_files)
File "/home/ahal/dev/mozilla-unified/python/mozbuild/mozbuild/vendor/vendor_python.py", line 54, in vendor
subprocess.check_call(
File "/home/ahal/.pyenv/versions/3.10.5/lib/python3.10/subprocess.py", line 369, in check_call
raise CalledProcessError(retcode, cmd)
Note for some reason the looseversion
and markupsafe
packages are concatenated together. Sylvestre also hit this error but with two different packages than me.
Reporter | ||
Comment 1•2 years ago
|
||
This specific error is happening because the looseversion
package doesn't have any hashes and the parsing logic that removes environment markers is tripping up over this fact. I have a patch to make the parsing logic more robust, but that just changes the error to:
ERROR: Hashes are required in --require-hashes mode, but they are missing from some requirements. Here is a list of those requirements along with the hashes their downloaded archives actually had. Add lines like these to your requirements files to prevent tampering. (If you did not enable --require-hashes manually, note that it turns on automatically when any package has a hash.)
looseversion==1.0.1 --hash=sha256:a205beabd0ffd40488edb9ccb3a39134510fc7c0c2847a25079f559e59c004ac
For whatever reason Poetry does not pin the hashes of the looseversion package. I notice that this dependency was added a few days ago in bug 1777414. This looks like an upstream Poetry problem, so I filed an issue.
Alex, fyi mach vendor
is now broken. I'm still not 100% sure what's going on, but we may want to consider backing out your patch (I can confirm that upgrading to the parent of your revision "fixes" mach vendor python
). Though this definitely isn't your fault or anything.. I'm just not sure how to fix this :).
Reporter | ||
Comment 2•2 years ago
|
||
Thanks to a tip from someone in the poetry
issue, clearing the cache seemed to work:
$ poetry cache clear pypi --all
Though now I'm hitting the same error that Sylvestre was hitting ><:
Writing lock file
ERROR: Could not find a version that satisfies the requirement aiohttp==3.7.4.post0appdirs==1.4.4 (from versions ...)
ERROR: No matching distribution found for aiohttp==3.7.4.post0appdirs==1.4.4
Error running mach:
['vendor', 'python']
I guess it's still progress?
Assignee | ||
Comment 3•2 years ago
|
||
I think this is the same root issue as in bug 1765416. (Upstream PyPi change causing a breakage in Poetry). The error message is slightly different, but I think it's closely related.
My changes up to and including D150814 fix this for me on Ubuntu and I can run ./mach vendor python
successfully. For now, if you want to vendor something, the best way to do it is just to pull down those changes (ideally on Linux) and vendor on top of that stack. I'm afraid I can't land the stack until we upgrade the images on some CI machines, otherwise we'll have a ton of failures in CI.
I have at least one other issue on with Poetry on Windows (in my downstream venv lockfile stuff), so right now I'm leaning towards restricting ./mach vendor python
to Linux only again. It used to be that way before, so I think it's reasonable to revert back to that. I do want to get the Windows Poetry problem(s) fixed, but it'll have to be upstream and I need to come up with a minimal test case first before I can report the issue.
Assignee | ||
Comment 4•2 years ago
|
||
(In reply to Andrew Halberstadt [:ahal] from comment #2)
Thanks to a tip from someone in the
poetry
issue, clearing the cache seemed to work:$ poetry cache clear pypi --all
That is the right command to fix the upstream PyPi problem, but you also need to upgrade to the newer Poetry for it to pull data correctly, or it'll just cache garbage again anyway and still fail ./mach vendor python
.
On Windows it's a bit trickier since if you don't have Poetry installed outside of the venv, that command will not work. When I was debugging this initially I had to manually go C:\Users\ahochheiden\AppData\Local\pypoetry
and delete the cache myself.
Assignee | ||
Comment 5•2 years ago
|
||
(In reply to Andrew Halberstadt [:ahal] from comment #1)
Alex, fyi
mach vendor
is now broken. I'm still not 100% sure what's going on, but we may want to consider backing out your patch (I can confirm that upgrading to the parent of your revision "fixes"mach vendor python
). Though this definitely isn't your fault or anything.. I'm just not sure how to fix this :).
It was broken before I landed that. See bug 1765416.
I will look into the looseversion
problem though. I do not have those changes on the base of my Poetry upgrade stack, and if it doesn't have hashes that's definitely a problem.
Reporter | ||
Comment 6•2 years ago
|
||
Thanks for all the context and patches! I can confirm that vendoring works if rebase on top of your stack :)
Assignee | ||
Comment 7•2 years ago
|
||
Perfect. I think the looseversion
problem you mentioned is just a symptom of the upstream PyPi change and older Poetry interacting, so no new problem there. I will do a proper rebase to that stack and push it so it's not annoying to deal with the merge conflicts for anyone else while this is in temporary limbo waiting for the Python3.7 bump.
Comment 8•2 years ago
|
||
This should be Regressed by: 1777414
Assignee | ||
Updated•2 years ago
|
Updated•2 years ago
|
Reporter | ||
Comment 9•2 years ago
|
||
Assignee | ||
Comment 10•2 years ago
|
||
Due to an upstream change on PyPi that makes the Poetry cache garbage,
this command is useful for after users upgrade to a newer version of
Poetry to remove the stale cached data, since in some cases it must be
done manually due to Poetry only being installed in a virtuelenv.
Updated•2 years ago
|
Assignee | ||
Comment 11•2 years ago
|
||
Depends on D153661
Assignee | ||
Comment 12•2 years ago
|
||
The easiest way to do was to write the text to the file as bytes. This
makes "\n" actually a "LF", instead of a "CR LF" while in text mode,
keeping the output consistent with POSIX systems.
Depends on D153662
Updated•2 years ago
|
Comment 13•2 years ago
|
||
I started looking again into having Updatebot update python packages automatically but I'm blocked by this, so I'm adding it to the deps.
Updated•2 years ago
|
Updated•2 years ago
|
Comment 15•2 years ago
|
||
Comment 16•2 years ago
|
||
bugherder |
https://hg.mozilla.org/mozilla-central/rev/a91335e84926
https://hg.mozilla.org/mozilla-central/rev/294c49621bf8
Updated•2 years ago
|
Assignee | ||
Comment 17•2 years ago
|
||
There are still more patches to land. This should stay open.
Assignee | ||
Comment 18•2 years ago
|
||
There is a bug in Poetry 1.2.0a2 caused by a breaking change
on PyPi's end: https://github.com/pypi/warehouse/pull/11775
This bug was fixed in Poetry 1.2.0b3, but 1.2.0b1 dropped
support for Python 3.6, which is a requirement for us.
As a temporary workaround, we can patch the fix into our
virtualenv's copy of Poetry. This patch should be removed
once we switch to using a newer version of Poetry.
Depends on D161538
Updated•2 years ago
|
Updated•2 years ago
|
Comment 20•2 years ago
|
||
Comment 21•2 years ago
|
||
bugherder |
Comment 22•2 years ago
|
||
Can we close this out now?
Comment 23•2 years ago
|
||
Doesn't seem so. I just tried with a Debian testing
$ ./mach vendor python
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
poetry 1.2.0a2 requires packaging<21.0,>=20.4, but you have packaging 21.3 which is incompatible.
Successfully installed poetry-core-1.1.0a6
Resolving dependencies... (0.1s)<debug>PackageInfo:</debug> Invalid constraint (0.6 ; extra == 'rq') found in sentry-sdk-0.14.3 dependencies, skipping
Resolving dependencies... (0.5s)
Writing lock file
ERROR: Could not find a version that satisfies the requirement aiohttp==3.7.4.post0appdirs==1.4.4 (from versions: 0.1, 0.2, 0.3, 0.4, 0.4.1, 0.4.2, 0.4.3, 0.4.4, 0.5.0, 0.6.0, 0.6.1, 0.6.2, 0.6.3, 0.6.4, 0.6.5, 0.7.0, 0.7.1, 0.7.2, 0.7.3, 0.8.0, 0.8.1, 0.8.2, 0.8.3, 0.8.4, 0.9.0, 0.9.1, 0.9.2, 0.9.3, 0.10.0, 0.10.1, 0.10.2, 0.11.0, 0.12.0, 0.13.0, 0.13.1, 0.14.0, 0.14.1, 0.14.2, 0.14.3, 0.14.4, 0.15.0, 0.15.1, 0.15.2, 0.15.3, 0.16.0, 0.16.1, 0.16.2, 0.16.3, 0.16.4, 0.16.5, 0.16.6, 0.17.0, 0.17.1, 0.17.2, 0.17.3, 0.17.4, 0.18.0, 0.18.1, 0.18.2, 0.18.3, 0.18.4, 0.19.0, 0.20.0, 0.20.1, 0.20.2, 0.21.0, 0.21.1, 0.21.2, 0.21.4, 0.21.5, 0.21.6, 0.22.0a0, 0.22.0b0, 0.22.0b1, 0.22.0b2, 0.22.0b3, 0.22.0b4, 0.22.0b5, 0.22.0b6, 0.22.0, 0.22.1, 0.22.2, 0.22.3, 0.22.4, 0.22.5, 1.0.0, 1.0.1, 1.0.2, 1.0.3, 1.0.5, 1.1.0, 1.1.1, 1.1.2, 1.1.3, 1.1.4, 1.1.5, 1.1.6, 1.2.0, 1.3.0, 1.3.1, 1.3.2, 1.3.3, 1.3.4, 1.3.5, 2.0.0rc1, 2.0.0, 2.0.1, 2.0.2, 2.0.3, 2.0.4, 2.0.5, 2.0.6.post1, 2.0.7, 2.1.0, 2.2.0, 2.2.1, 2.2.2, 2.2.3, 2.2.4, 2.2.5, 2.3.0a4, 2.3.0, 2.3.1, 2.3.2b2, 2.3.2b3, 2.3.2, 2.3.3, 2.3.4, 2.3.5, 2.3.6, 2.3.7, 2.3.8, 2.3.9, 2.3.10, 3.0.0b0, 3.0.0b1, 3.0.0b2, 3.0.0b3, 3.0.0b4, 3.0.0, 3.0.1, 3.0.2, 3.0.3, 3.0.4, 3.0.5, 3.0.6, 3.0.7, 3.0.8, 3.0.9, 3.1.0, 3.1.1, 3.1.2, 3.1.3, 3.2.0, 3.2.1, 3.3.0a0, 3.3.0, 3.3.1, 3.3.2a0, 3.3.2, 3.4.0a0, 3.4.0a3, 3.4.0b1, 3.4.0b2, 3.4.0, 3.4.1, 3.4.2, 3.4.3, 3.4.4, 3.5.0a1, 3.5.0b1, 3.5.0b2, 3.5.0b3, 3.5.0, 3.5.1, 3.5.2, 3.5.3, 3.5.4, 3.6.0a0, 3.6.0a1, 3.6.0a2, 3.6.0a3, 3.6.0a4, 3.6.0a5, 3.6.0a6, 3.6.0a7, 3.6.0a8, 3.6.0a9, 3.6.0a11, 3.6.0a12, 3.6.0b0, 3.6.0, 3.6.1b3, 3.6.1b4, 3.6.1, 3.6.2a0, 3.6.2a1, 3.6.2a2, 3.6.2, 3.6.3, 3.7.0b0, 3.7.0b1, 3.7.0, 3.7.1, 3.7.2, 3.7.3, 3.7.4, 3.7.4.post0, 3.8.0a7, 3.8.0b0, 3.8.0, 3.8.1, 3.8.2, 3.8.3, 4.0.0a0, 4.0.0a1)
ERROR: No matching distribution found for aiohttp==3.7.4.post0appdirs==1.4.4
obsolete feature not enabled but 849 markers found!
Error running mach:
['vendor', 'python']
The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.
You can invoke |./mach busted| to check if this issue is already on file. If it
isn't, please use |./mach busted file vendor| to report it. If |./mach busted| is
misbehaving, you can also inspect the dependencies of bug 1543241.
If filing a bug, please include the full output of mach, including this error
message.
The details of the failure are as follows:
subprocess.CalledProcessError: Command '['/home/sylvestre/dev/mozilla/mozilla-central.hg/obj-x86_64-pc-linux-gnu/_virtualenvs/vendor/bin/python', '-m', 'pip', 'download', '-r', '/tmp/tmpo90eq05x/requirements.no-markers.txt', '--no-deps', '--dest', '/tmp/tmp9n2cmkan', '--abi', 'none', '--platform', 'any']' returned non-zero exit status 1.
File "/home/sylvestre/dev/mozilla/mozilla-central.hg/python/mozbuild/mozbuild/vendor/mach_commands.py", line 229, in vendor_python
vendor_command.vendor(keep_extra_files)
File "/home/sylvestre/dev/mozilla/mozilla-central.hg/python/mozbuild/mozbuild/vendor/vendor_python.py", line 54, in vendor
subprocess.check_call(
File "/usr/lib/python3.10/subprocess.py", line 369, in check_call
raise CalledProcessError(retcode, cmd)
Assignee | ||
Comment 24•2 years ago
|
||
(In reply to Jim Mathies [:jimm] from comment #22)
Can we close this out now?
The current fix is just a temporary one. I want to keep this open until I can implement the permanent solution (Blocked until Python 3.7 on all CI machines).
(In reply to Sylvestre Ledru [:Sylvestre] from comment #23)
Doesn't seem so. I just tried with a Debian testing
I just tested on Debian 11.3 and it works fine for me. Can you try clearing your Poetry cache? (and pip cache if that doesn't work)
./mach python --virtualenv=vendor -m poetry cache clear pypi --all
pip3 cache purge
Comment 25•2 years ago
|
||
I just tested on Debian 11.3 and it works fine for me. Can you try clearing your Poetry cache? (and pip cache if that doesn't work)
./mach python --virtualenv=vendor -m poetry cache clear pypi --all pip3 cache purge
\o/
It works.
Maybe we should suggest what you said in the error message?
./mach python --virtualenv=vendor -m poetry cache clear pypi --all
pip3 cache purge
Assignee | ||
Updated•2 years ago
|
Comment 26•2 years ago
|
||
The Bugbug bot thinks this bug should belong to the 'Firefox Build System::Mach Core' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.
Assignee | ||
Comment 27•2 years ago
|
||
This version is compatible with poetry
1.4.0
and removes blockers for various other ongoing projects.
https://github.com/python/importlib_metadata/blob/main/CHANGES.rst#v660
Depends on D149679
Assignee | ||
Comment 28•2 years ago
|
||
https://github.com/python-poetry/poetry/releases/tag/1.4.0https://github.com/python-poetry/poetry/releases/tag/1.4.0
https://github.com/python-poetry/poetry-core/releases/tag/1.5.1
The changes in third_party/python/requirements.txt
, third_party/python/poetry.lock
and mozbuild/test/vendor_requirements.txt
are due to the newer version.
Depends on D150813
Assignee | ||
Comment 29•2 years ago
|
||
Depends on D171761
Comment 30•2 years ago
|
||
Comment 31•2 years ago
|
||
bugherder |
Updated•1 year ago
|
Assignee | ||
Updated•1 year ago
|
Updated•1 year ago
|
Updated•1 year ago
|
Description
•