Closed Bug 1665675 Opened 4 years ago Closed 4 years ago

"AttributeError: module distutils has no attribute sysconfig" when building Firefox

Categories

(Firefox Build System :: General, defect, P3)

defect

Tracking

(firefox-esr68 unaffected, firefox-esr78 fixed, firefox80 unaffected, firefox81 unaffected, firefox82 wontfix, firefox83 fixed)

RESOLVED FIXED
83 Branch
Tracking Status
firefox-esr68 --- unaffected
firefox-esr78 --- fixed
firefox80 --- unaffected
firefox81 --- unaffected
firefox82 --- wontfix
firefox83 --- fixed

People

(Reporter: sefeng, Assigned: rstewart)

References

(Regression)

Details

(Keywords: regression)

Attachments

(1 file)

This is something that we fixed in bug 1662130, however it came back after bug 1663755.

 0:04.02   File "/home/sefeng/.local/workspace/mozilla/mozilla-unified4/build/moz.configure/init.configure", line 243, in <module>
 0:04.02     def virtualenv_python3(env_python, virtualenv_name, build_env, mozconfig, help):
 0:04.02   File "/home/sefeng/.local/workspace/mozilla/mozilla-unified4/python/mozbuild/mozbuild/configure/__init__.py", line 758, in decorator
 0:04.03     depends = DependsFunction(self, func, dependencies, when=when)
 0:04.03   File "/home/sefeng/.local/workspace/mozilla/mozilla-unified4/python/mozbuild/mozbuild/configure/__init__.py", line 129, in __init__
 0:04.03     sandbox._value_for(self)
 0:04.03   File "/home/sefeng/.local/workspace/mozilla/mozilla-unified4/python/mozbuild/mozbuild/configure/__init__.py", line 544, in _value_for
 0:04.03     return self._value_for_depends(obj)
 0:04.03   File "/home/sefeng/.local/workspace/mozilla/mozilla-unified4/python/mozbuild/mozbuild/util.py", line 1021, in method_call
 0:04.03     cache[args] = self.func(instance, *args)
 0:04.03   File "/home/sefeng/.local/workspace/mozilla/mozilla-unified4/python/mozbuild/mozbuild/configure/__init__.py", line 553, in _value_for_depends
 0:04.03     value = obj.result()
 0:04.03   File "/home/sefeng/.local/workspace/mozilla/mozilla-unified4/python/mozbuild/mozbuild/util.py", line 1021, in method_call
 0:04.03     cache[args] = self.func(instance, *args)
 0:04.03   File "/home/sefeng/.local/workspace/mozilla/mozilla-unified4/python/mozbuild/mozbuild/configure/__init__.py", line 155, in result
 0:04.03     return self._func(*resolved_args)
 0:04.03   File "/home/sefeng/.local/workspace/mozilla/mozilla-unified4/python/mozbuild/mozbuild/configure/__init__.py", line 1164, in wrapped
 0:04.03     return new_func(*args, **kwargs)
 0:04.03   File "/home/sefeng/.local/workspace/mozilla/mozilla-unified4/build/moz.configure/init.configure", line 371, in virtualenv_python3
 0:04.03     if not distutils.sysconfig.get_python_lib():
 0:04.03 AttributeError: module 'distutils' has no attribute 'sysconfig'
 0:04.04 *** Fix above errors and then restart with\
 0:04.04                "./mach build"
 0:04.04 make: *** [client.mk:115: configure] Error 1

I am running Manjaro, a arch based distro.

With the latest version of virtualenv I can reproduce this on Ubuntu. I think it's a bug in the configure sandbox.

Assignee: nobody → rstewart
Severity: -- → S3
Priority: -- → P3

The existing implementation of @imports() in the configure sandbox doesn't translate an import of the form @imports('distutils.sysconfig') into an import distutils.sysconfig statement; instead, it transforms the input @imports() request a few times in such a way that we eventually just do import distutils, and expect that distutils.sysconfig will be populated that way. This would be fine, except that this isn't the way that Python's import system works:

>>> import distutils
>>> distutils.sysconfig
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'distutils' has no attribute 'sysconfig'
>>> import distutils.sysconfig
>>> distutils.sysconfig
<module 'distutils.sysconfig' from '/usr/lib/python3.8/distutils/sysconfig.py'>

i.e., we can't just import a parent module and expect that we can indirectly access all child packages of that module without importing them specifically.

So instead, we simplify the current model somewhat by not transforming the @imports() request at all and instead just performing the exact import that the user requested. This resolves the distutils.sysconfig issue as well as hopefully preventing any other similar issues popping up in the future.

While I'm here, I also refactored some stuff so that the way that we're patching in wrapped modules for the sandbox is more structured.

Set release status flags based on info from the regressing bug 1663755

D90627 seems to have fixed this issue for me. I'm running Manjaro w/ Python 3.8.5 & viritualenv 20.0.23.

I encountered this problem, and independently reached the same conclusion as written in comment 2.

The @imports('distutils.sysconfig') decorator is incorrectly translated to import distutils as imported.

The patch from comment 2 looks like the desired general solution. Given the constraints I unbroke my build with the following change:

diff --git a/build/moz.configure/init.configure b/build/moz.configure/init.configure
index 970ef849e5e9..bc74d1508c56 100644
--- a/build/moz.configure/init.configure
+++ b/build/moz.configure/init.configure
@@ -233,7 +233,7 @@ option(env='VIRTUALENV_NAME', nargs=1, default='init_py3',
 @imports('os')
 @imports('sys')
 @imports('subprocess')
-@imports('distutils.sysconfig')
+@imports(_from='distutils.sysconfig', _import='get_python_lib')
 @imports(_from='mozbuild.configure.util', _import='LineIO')
 @imports(_from='mozbuild.virtualenv', _import='VirtualenvManager')
 @imports(_from='mozbuild.virtualenv', _import='verify_python_version')
@@ -368,7 +368,7 @@ def virtualenv_python3(env_python, virtualenv_name, build_env, mozconfig, help):
         sys.exit(subprocess.call([python] + sys.argv))
 
     # We are now in the virtualenv
-    if not distutils.sysconfig.get_python_lib():
+    if not get_python_lib():
         die('Could not determine python site packages directory')
 
     str_version = '.'.join(str(v) for v in version)

That patch fixed for me with Arch Linux

Blocks: 1654457

The patch worked for me on VoidLinux.

Pushed by rstewart@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/37d14279b86b Fix sporadic `AttributeError: module distutils has no attribute sysconfig` error in `configure` r=glandium
Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
Target Milestone: --- → 83 Branch
Regressed by: 1654457
Has Regression Range: --- → yes

Comment on attachment 9176404 [details]
Bug 1665675 - Fix sporadic AttributeError: module distutils has no attribute sysconfig error in configure

ESR Uplift Approval Request

  • If this is not a sec:{high,crit} bug, please state case for ESR consideration: Resolves configure regression from 1654457
  • User impact if declined: Local builds fail during configure
  • Fix Landed on Version: 83
  • Risk to taking this patch: Low
  • Why is the change risky/not risky? (and alternatives if risky): Build patch
  • String or UUID changes made by this patch:
Attachment #9176404 - Flags: approval-mozilla-esr78?

Comment on attachment 9176404 [details]
Bug 1665675 - Fix sporadic AttributeError: module distutils has no attribute sysconfig error in configure

Needed to better support running mach on newer macOS releases. Approved for 78.8esr.

Attachment #9176404 - Flags: approval-mozilla-esr78? → approval-mozilla-esr78+
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: