"AttributeError: module distutils has no attribute sysconfig" when building Firefox
Categories
(Firefox Build System :: General, defect, P3)
Tracking
(firefox-esr68 unaffected, firefox-esr78 fixed, firefox80 unaffected, firefox81 unaffected, firefox82 wontfix, firefox83 fixed)
Tracking | Status | |
---|---|---|
firefox-esr68 | --- | unaffected |
firefox-esr78 | --- | fixed |
firefox80 | --- | unaffected |
firefox81 | --- | unaffected |
firefox82 | --- | wontfix |
firefox83 | --- | fixed |
People
(Reporter: sefeng, Assigned: rstewart)
References
(Regression)
Details
(Keywords: regression)
Attachments
(1 file)
(deleted),
text/x-phabricator-request
|
RyanVM
:
approval-mozilla-esr78+
|
Details |
This is something that we fixed in bug 1662130, however it came back after bug 1663755.
0:04.02 File "/home/sefeng/.local/workspace/mozilla/mozilla-unified4/build/moz.configure/init.configure", line 243, in <module>
0:04.02 def virtualenv_python3(env_python, virtualenv_name, build_env, mozconfig, help):
0:04.02 File "/home/sefeng/.local/workspace/mozilla/mozilla-unified4/python/mozbuild/mozbuild/configure/__init__.py", line 758, in decorator
0:04.03 depends = DependsFunction(self, func, dependencies, when=when)
0:04.03 File "/home/sefeng/.local/workspace/mozilla/mozilla-unified4/python/mozbuild/mozbuild/configure/__init__.py", line 129, in __init__
0:04.03 sandbox._value_for(self)
0:04.03 File "/home/sefeng/.local/workspace/mozilla/mozilla-unified4/python/mozbuild/mozbuild/configure/__init__.py", line 544, in _value_for
0:04.03 return self._value_for_depends(obj)
0:04.03 File "/home/sefeng/.local/workspace/mozilla/mozilla-unified4/python/mozbuild/mozbuild/util.py", line 1021, in method_call
0:04.03 cache[args] = self.func(instance, *args)
0:04.03 File "/home/sefeng/.local/workspace/mozilla/mozilla-unified4/python/mozbuild/mozbuild/configure/__init__.py", line 553, in _value_for_depends
0:04.03 value = obj.result()
0:04.03 File "/home/sefeng/.local/workspace/mozilla/mozilla-unified4/python/mozbuild/mozbuild/util.py", line 1021, in method_call
0:04.03 cache[args] = self.func(instance, *args)
0:04.03 File "/home/sefeng/.local/workspace/mozilla/mozilla-unified4/python/mozbuild/mozbuild/configure/__init__.py", line 155, in result
0:04.03 return self._func(*resolved_args)
0:04.03 File "/home/sefeng/.local/workspace/mozilla/mozilla-unified4/python/mozbuild/mozbuild/configure/__init__.py", line 1164, in wrapped
0:04.03 return new_func(*args, **kwargs)
0:04.03 File "/home/sefeng/.local/workspace/mozilla/mozilla-unified4/build/moz.configure/init.configure", line 371, in virtualenv_python3
0:04.03 if not distutils.sysconfig.get_python_lib():
0:04.03 AttributeError: module 'distutils' has no attribute 'sysconfig'
0:04.04 *** Fix above errors and then restart with\
0:04.04 "./mach build"
0:04.04 make: *** [client.mk:115: configure] Error 1
I am running Manjaro, a arch based distro.
Reporter | ||
Updated•4 years ago
|
Assignee | ||
Comment 1•4 years ago
|
||
With the latest version of virtualenv
I can reproduce this on Ubuntu. I think it's a bug in the configure
sandbox.
Assignee | ||
Comment 2•4 years ago
|
||
The existing implementation of @imports()
in the configure
sandbox doesn't translate an import of the form @imports('distutils.sysconfig')
into an import distutils.sysconfig
statement; instead, it transforms the input @imports()
request a few times in such a way that we eventually just do import distutils
, and expect that distutils.sysconfig
will be populated that way. This would be fine, except that this isn't the way that Python's import
system works:
>>> import distutils
>>> distutils.sysconfig
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: module 'distutils' has no attribute 'sysconfig'
>>> import distutils.sysconfig
>>> distutils.sysconfig
<module 'distutils.sysconfig' from '/usr/lib/python3.8/distutils/sysconfig.py'>
i.e., we can't just import a parent module and expect that we can indirectly access all child packages of that module without importing them specifically.
So instead, we simplify the current model somewhat by not transforming the @imports()
request at all and instead just performing the exact import
that the user requested. This resolves the distutils.sysconfig
issue as well as hopefully preventing any other similar issues popping up in the future.
While I'm here, I also refactored some stuff so that the way that we're patching in wrapped modules for the sandbox is more structured.
Comment 3•4 years ago
|
||
Set release status flags based on info from the regressing bug 1663755
Comment 4•4 years ago
|
||
D90627 seems to have fixed this issue for me. I'm running Manjaro w/ Python 3.8.5 & viritualenv 20.0.23.
Updated•4 years ago
|
Comment 5•4 years ago
|
||
I encountered this problem, and independently reached the same conclusion as written in comment 2.
The @imports('distutils.sysconfig')
decorator is incorrectly translated to import distutils as imported
.
The patch from comment 2 looks like the desired general solution. Given the constraints I unbroke my build with the following change:
diff --git a/build/moz.configure/init.configure b/build/moz.configure/init.configure
index 970ef849e5e9..bc74d1508c56 100644
--- a/build/moz.configure/init.configure
+++ b/build/moz.configure/init.configure
@@ -233,7 +233,7 @@ option(env='VIRTUALENV_NAME', nargs=1, default='init_py3',
@imports('os')
@imports('sys')
@imports('subprocess')
-@imports('distutils.sysconfig')
+@imports(_from='distutils.sysconfig', _import='get_python_lib')
@imports(_from='mozbuild.configure.util', _import='LineIO')
@imports(_from='mozbuild.virtualenv', _import='VirtualenvManager')
@imports(_from='mozbuild.virtualenv', _import='verify_python_version')
@@ -368,7 +368,7 @@ def virtualenv_python3(env_python, virtualenv_name, build_env, mozconfig, help):
sys.exit(subprocess.call([python] + sys.argv))
# We are now in the virtualenv
- if not distutils.sysconfig.get_python_lib():
+ if not get_python_lib():
die('Could not determine python site packages directory')
str_version = '.'.join(str(v) for v in version)
Comment 7•4 years ago
|
||
The patch worked for me on VoidLinux.
Comment 9•4 years ago
|
||
bugherder |
Updated•4 years ago
|
Updated•4 years ago
|
Comment 10•4 years ago
|
||
Comment on attachment 9176404 [details]
Bug 1665675 - Fix sporadic AttributeError: module distutils has no attribute sysconfig
error in configure
ESR Uplift Approval Request
- If this is not a sec:{high,crit} bug, please state case for ESR consideration: Resolves
configure
regression from 1654457 - User impact if declined: Local builds fail during
configure
- Fix Landed on Version: 83
- Risk to taking this patch: Low
- Why is the change risky/not risky? (and alternatives if risky): Build patch
- String or UUID changes made by this patch:
Comment 11•4 years ago
|
||
Comment on attachment 9176404 [details]
Bug 1665675 - Fix sporadic AttributeError: module distutils has no attribute sysconfig
error in configure
Needed to better support running mach on newer macOS releases. Approved for 78.8esr.
Comment 12•4 years ago
|
||
bugherder uplift |
Updated•4 years ago
|
Description
•