./mach lint does not work on MacOS 10.15 due to Python 3 virtualenv errors
Categories
(Firefox Build System :: General, defect)
Tracking
(Not tracked)
People
(Reporter: tjr, Unassigned)
References
(Blocks 1 open bug)
Details
Attachments
(2 files)
Trying to run ./mach lint results in the following error:
./mach lint testing/mochitest/runtests.py
Using base prefix '/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7'
/Users/tritter/Documents/moz/mozilla-unified/third_party/python/virtualenv/virtualenv.py:1047: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import imp
New python executable in /Users/tritter/Documents/moz/mozilla-unified/obj-x86_64-apple-darwin18.5.0.opt/_virtualenvs/init_py3/bin/python3
Also creating executable in /Users/tritter/Documents/moz/mozilla-unified/obj-x86_64-apple-darwin18.5.0.opt/_virtualenvs/init_py3/bin/python
Traceback (most recent call last):
File "/Users/tritter/Documents/moz/mozilla-unified/third_party/python/virtualenv/virtualenv.py", line 2349, in <module>
main()
File "/Users/tritter/Documents/moz/mozilla-unified/third_party/python/virtualenv/virtualenv.py", line 712, in main
symlink=options.symlink)
File "/Users/tritter/Documents/moz/mozilla-unified/third_party/python/virtualenv/virtualenv.py", line 927, in create_environment
site_packages=site_packages, clear=clear, symlink=symlink))
File "/Users/tritter/Documents/moz/mozilla-unified/third_party/python/virtualenv/virtualenv.py", line 1342, in install_python
shutil.copy(original_python, py_executable)
UnboundLocalError: local variable 'original_python' referenced before assignment
Error running mach:
['lint', 'testing/mochitest/runtests.py']
The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.
You can invoke |./mach busted| to check if this issue is already on file. If it
isn't, please use |./mach busted file| to report it. If |./mach busted| is
misbehaving, you can also inspect the dependencies of bug 1543241.
If filing a bug, please include the full output of mach, including this error
message.
The details of the failure are as follows:
Exception: Failed to create virtualenv: /Users/tritter/Documents/moz/mozilla-unified/obj-x86_64-apple-darwin18.5.0.opt/_virtualenvs/init_py3
File "/Users/tritter/Documents/moz/mozilla-unified/tools/lint/mach_commands.py", line 63, in lint
self._activate_virtualenv()
File "/Users/tritter/Documents/moz/mozilla-unified/python/mozbuild/mozbuild/base.py", line 818, in _activate_virtualenv
self.virtualenv_manager.ensure()
File "/Users/tritter/Documents/moz/mozilla-unified/python/mozbuild/mozbuild/virtualenv.py", line 178, in ensure
return self.build(python)
File "/Users/tritter/Documents/moz/mozilla-unified/python/mozbuild/mozbuild/virtualenv.py", line 477, in build
self.create(python)
File "/Users/tritter/Documents/moz/mozilla-unified/python/mozbuild/mozbuild/virtualenv.py", line 217, in create
'Failed to create virtualenv: %s' % self.virtualenv_root)
If I remove obj-x86_64-apple-darwin18.5.0.opt/_virtualenvs/
and run it again, I get the same error.
I have a fresh mach bootstrap, I just updated and rebased central, and my mozconfig is:
ac_add_options --with-macos-sdk=/Users/tritter/Documents/src/osx-sdk/MacOSX10.11.sdk
ac_add_options --enable-debug
mk_add_options MOZ_OBJDIR=obj-x86_64-apple-darwin18.5.0.opt
ac_add_options --enable-optimize
Comment 1•5 years ago
|
||
This is one of those fun bugs where Google returns a single hit: https://github.com/pypa/virtualenv/issues/750.
Could you please rerun the mach command with the environment variable _VIRTUALENV_DEBUG
set to 1? virtualenv.py should dump a bunch of extra information to the console.
$ env _VIRTUALENV_DEBUG=1 ./mach lint testing/mochitest/runtests.py
Updated•5 years ago
|
Reporter | ||
Comment 2•5 years ago
|
||
Comment 3•5 years ago
|
||
The version of virtualenv.py bundled in the Firefox source tree is out of date for the development tools that you are using. We need to upgrade to virtualenv.py v16.7.6.
Issue: https://github.com/pypa/virtualenv/issues/1402
Fixed by: https://github.com/pypa/virtualenv/pull/1425
Release: https://virtualenv.pypa.io/en/latest/changes/#v16-7-6-2019-10-16
Comment 4•5 years ago
|
||
We are on virtualenv v15.2.0: https://searchfox.org/mozilla-central/source/third_party/python/requirements.txt#121
Updated•5 years ago
|
Comment 5•5 years ago
|
||
Comment 6•5 years ago
|
||
(In reply to Sebastian Zartner [:sebo] from comment #5)
For what it's worth, I have the same issue on Windows 10. See the attached
outputSebastian
Thanks for the report. That's a different error, "AssertionError: SRE module mismatch". Could you please file a new bug for it?
Comment 7•5 years ago
|
||
Ah, didn't realize that it's a different reason. Thanks for the fast response! I've filed bug 1599244 for it now.
Sebastian
Comment 8•5 years ago
|
||
We've found a workaround for this bug. It only affects developers who are building Firefox with the Xcode-bundled Python 3. Developers who install Python 3 from Homebrew and set $PATH
appropriately are not affected. See bug 1600720 comment 7 for details.
Comment 9•5 years ago
|
||
:tjr, a fix for this has landed in mozilla-central (see bug 1464038). If you are still building with Xcode could you please try pulling the latest mozilla-central and see if ./mach lint
works?
Comment 10•5 years ago
|
||
@Ricky Are you able to verify this bugfix on your system? It should be possible by setting $PATH to the Python 3 framework installation. env PATH=/path/to/Python3.framework/some/bin ./mach lint
.
Comment 11•5 years ago
|
||
Unfortunately I don't seem to have a Python3.framework
directory on my machine.
Updated•5 years ago
|
Description
•