I'm surely doing something wrong here too but running ./mach build with MozillaBuild 3.1 after a clean install and clobber, I get: 0:01.01 c:\mozilla-build\mozmake\mozmake.EXE -f client.mk -s configure 0:04.23 Clobber not needed. 0:07.31 cd c:/moz/src1/obj-debug-opt 0:07.34 c:/moz/src1/configure 0:07.88 Creating Python environment 0:17.47 New python executable in c:\moz\src1\obj-debug-opt\_virtualenv\Scripts\python2.7.exe 0:17.47 Also creating executable in c:\moz\src1\obj-debug-opt\_virtualenv\Scripts\python.exe 0:17.47 Installing setuptools, pip, wheel...done. 0:17.88 running build_ext 0:17.88 0:17.88 building 'psutil._psutil_windows' extension 0:17.88 0:17.88 error: Microsoft Visual C++ 9.0 is required. Get it from http://aka.ms/vcpython27 0:17.88 0:17.88 0:17.88 Error processing command. Ignoring because optional. (optional:setup.py:third_party/python/psutil:build_ext:--inplace) 0:17.88 Error processing command. Ignoring because optional. (optional:packages.txt:comm/build/virtualenv_packages.txt) 0:17.88 c:\moz\src1\python\mozbuild\mozbuild\virtualenv.py:376: UserWarning: Hacking environment to allow binary Python extensions to build. You can make this warning go away by installing Visual Studio 2008. You can download the Express Edition installer from http://go.microsoft.com/?linkid=7729279 0:17.88 warnings.warn('Hacking environment to allow binary Python ' 0:17.88 Reexecuting in the virtualenv 0:18.33 Adding configure options from c:\Users\Brian\mozconfig\debug-opt 0:18.33 --enable-application=browser 0:18.33 --enable-optimize 0:18.34 --enable-debug 0:18.34 --enable-tests 0:18.34 --target=x86_64-pc-mingw32 0:18.34 --host=x86_64-pc-mingw32 0:18.34 checking for vcs source checkout... hg 0:18.58 checking for a shell... c:/mozilla-build/msys/bin/sh.exe 0:18.86 checking for host system type... x86_64-pc-mingw32 0:19.17 checking for target system type... x86_64-pc-mingw32 0:19.25 checking for a shell... c:/mozilla-build/msys/bin/sh.exe 0:19.53 checking for host system type... x86_64-pc-mingw32 0:19.81 checking for target system type... x86_64-pc-mingw32 0:19.85 checking for vcs source checkout... hg 0:19.85 checking whether cross compiling... no 0:19.94 checking for the target C compiler... 'C:/PROGRA~2/MIB055~1/2017/PROFES~1/VC/Tools/MSVC/1411~1.255/bin/HostX64/x64/cl.exe' 0:20.06 checking whether the target C compiler can be used... yes 0:20.12 checking for Python 3... c:\mozilla-build\python3\python3.EXE (3.6.3) 0:20.12 checking for hg... c:/mozilla-build/python/Scripts/hg.exe 0:20.30 checking for Mercurial version... 0:20.30 DEBUG: <truncated - see config.log for full output> 0:20.31 DEBUG: | self._load() 0:20.31 DEBUG: | File "c:\mozilla-build\python\Lib\site-packages\hgdemandimport\demandimportpy2.py", line 90, in _load 0:20.31 DEBUG: | mod = _hgextimport(_origimport, head, globals, locals, None, level) 0:20.31 DEBUG: | File "c:\mozilla-build\python\Lib\site-packages\hgdemandimport\demandimportpy2.py", line 41, in _hgextimport 0:20.31 DEBUG: | return importfunc(name, globals, *args, **kwargs) 0:20.31 DEBUG: | File "c:\mozilla-build\python\Lib\site-packages\mercurial\win32.py", line 22, in <module> 0:20.31 DEBUG: | _kernel32 = ctypes.windll.kernel32 0:20.31 DEBUG: | File "c:\mozilla-build\python\Lib\site-packages\hgdemandimport\demandimportpy2.py", line 145, in __getattr__ 0:20.31 DEBUG: | self._load() 0:20.31 DEBUG: | File "c:\mozilla-build\python\Lib\site-packages\hgdemandimport\demandimportpy2.py", line 90, in _load 0:20.31 DEBUG: | mod = _hgextimport(_origimport, head, globals, locals, None, level) 0:20.31 DEBUG: | File "c:\mozilla-build\python\Lib\site-packages\hgdemandimport\demandimportpy2.py", line 41, in _hgextimport 0:20.31 DEBUG: | return importfunc(name, globals, *args, **kwargs) 0:20.31 DEBUG: | File "c:\mozilla-build\python\lib\ctypes\__init__.py", line 7, in <module> 0:20.31 DEBUG: | from _ctypes import Union, Structure, Array 0:20.31 DEBUG: | File "c:\mozilla-build\python\Lib\site-packages\hgdemandimport\demandimportpy2.py", line 268, in _demandimport 0:20.31 DEBUG: | mod = _hgextimport(_origimport, name, globals, locals) 0:20.31 DEBUG: | File "c:\mozilla-build\python\Lib\site-packages\hgdemandimport\demandimportpy2.py", line 41, in _hgextimport 0:20.31 DEBUG: | return importfunc(name, globals, *args, **kwargs) 0:20.31 DEBUG: | ImportError: DLL load failed: 指定されたプロシージャが見つかりません。 0:20.31 ERROR: Command `c:/mozilla-build/python/Scripts/hg.exe --version` failed with exit status 1. 0:20.35 *** Fix above errors and then restart with\ 0:20.35 "c:/mozilla-build/mozmake/mozmake.EXE -f client.mk build" 0:20.37 mozmake.EXE: *** [client.mk:293: configure] Error 1 (The Japanese error there reads, "The specified procedure could not be found") Running: c:/mozilla-build/python/Scripts/hg.exe --version directly produces the same problem. `python --version` returns "Python 2.7.14" Re-running "./mach build" I get a slightly abbreviated log: 0:00.89 c:\mozilla-build\mozmake\mozmake.EXE -f client.mk -s configure 0:03.67 cd c:/moz/src1/obj-debug-opt 0:03.69 c:/moz/src1/configure 0:04.22 Reexecuting in the virtualenv 0:04.64 Adding configure options from c:\Users\Brian\mozconfig\debug-opt 0:04.64 --enable-application=browser 0:04.64 --enable-optimize 0:04.64 --enable-debug 0:04.64 --enable-tests 0:04.65 --target=x86_64-pc-mingw32 0:04.65 --host=x86_64-pc-mingw32 0:04.65 checking for vcs source checkout... hg 0:04.89 checking for a shell... c:/mozilla-build/msys/bin/sh.exe 0:05.17 checking for host system type... x86_64-pc-mingw32 0:05.45 checking for target system type... x86_64-pc-mingw32 0:05.51 checking for a shell... c:/mozilla-build/msys/bin/sh.exe 0:05.78 checking for host system type... x86_64-pc-mingw32 0:06.06 checking for target system type... x86_64-pc-mingw32 0:06.10 checking for vcs source checkout... hg 0:06.10 checking whether cross compiling... no 0:06.18 checking for the target C compiler... 'C:/PROGRA~2/MIB055~1/2017/PROFES~1/VC/Tools/MSVC/1411~1.255/bin/HostX64/x64/cl.exe' 0:06.31 checking whether the target C compiler can be used... yes 0:06.37 checking for Python 3... c:\mozilla-build\python3\python3.EXE (3.6.3) 0:06.37 checking for hg... c:/mozilla-build/python/Scripts/hg.exe 0:06.44 checking for Mercurial version... 0:06.44 DEBUG: <truncated - see config.log for full output> 0:06.44 DEBUG: | self._load() 0:06.44 DEBUG: | File "c:\mozilla-build\python\Lib\site-packages\hgdemandimport\demandimportpy2.py", line 90, in _load 0:06.44 DEBUG: | mod = _hgextimport(_origimport, head, globals, locals, None, level) 0:06.44 DEBUG: | File "c:\mozilla-build\python\Lib\site-packages\hgdemandimport\demandimportpy2.py", line 41, in _hgextimport 0:06.44 DEBUG: | return importfunc(name, globals, *args, **kwargs) 0:06.44 DEBUG: | File "c:\mozilla-build\python\Lib\site-packages\mercurial\win32.py", line 22, in <module> 0:06.44 DEBUG: | _kernel32 = ctypes.windll.kernel32 0:06.44 DEBUG: | File "c:\mozilla-build\python\Lib\site-packages\hgdemandimport\demandimportpy2.py", line 145, in __getattr__ 0:06.44 DEBUG: | self._load() 0:06.44 DEBUG: | File "c:\mozilla-build\python\Lib\site-packages\hgdemandimport\demandimportpy2.py", line 90, in _load 0:06.44 DEBUG: | mod = _hgextimport(_origimport, head, globals, locals, None, level) 0:06.44 DEBUG: | File "c:\mozilla-build\python\Lib\site-packages\hgdemandimport\demandimportpy2.py", line 41, in _hgextimport 0:06.44 DEBUG: | return importfunc(name, globals, *args, **kwargs) 0:06.44 DEBUG: | File "c:\mozilla-build\python\lib\ctypes\__init__.py", line 7, in <module> 0:06.44 DEBUG: | from _ctypes import Union, Structure, Array 0:06.44 DEBUG: | File "c:\mozilla-build\python\Lib\site-packages\hgdemandimport\demandimportpy2.py", line 268, in _demandimport 0:06.44 DEBUG: | mod = _hgextimport(_origimport, name, globals, locals) 0:06.44 DEBUG: | File "c:\mozilla-build\python\Lib\site-packages\hgdemandimport\demandimportpy2.py", line 41, in _hgextimport 0:06.44 DEBUG: | return importfunc(name, globals, *args, **kwargs) 0:06.44 DEBUG: | ImportError: DLL load failed: 指定されたプロシージャが見つかりません。 0:06.44 ERROR: Command `c:/mozilla-build/python/Scripts/hg.exe --version` failed with exit status 1. 0:06.49 *** Fix above errors and then restart with\ 0:06.49 "c:/mozilla-build/mozmake/mozmake.EXE -f client.mk build" 0:06.51 mozmake.EXE: *** [client.mk:293: configure] Error 1

Tried to debug this with RyanVM. Downgrading the version of mercurial doesn't seem to help. On the other hand, running `c:/mozilla-build/python/Scripts/hg.exe --version` from MozillaBuild 3.0 works, even after upgrading mercurial. We suspect it might be something in the Python environment that has changed. Furthermore, since no-one else since to have encountered this yet it might be due to the fact that I am on Japanese Windows and there are likely Japanese characters (probably Shift-JIS encoded, see bug 1414060) in some paths / environment variables. Ryan suggested gps might have some clues? The Python changelog includes some interesting entries such as: * Issue #29082: Fixed loading libraries in ctypes by unicode names on Windows * Issue #23908: os functions, open() and the io.FileIO constructor now reject unicode paths with embedded null character on Windows instead of silently truncating them

Ultimately, I'm hoping this is a behavior change in Python 2.7.14 that Mercurial can be updated to work around. Otherwise, our options start getting ugly in a hurry :\

Dropping the [extensions] section of my .hgrc doesn't seem to help.

I definitely don't see this. And I just switched the default language to Japanese, and didn't encounter this either. I did see something weird with Python environment, which doesn't seem to be related to mercurial, though. That weirdness got resolved after I clobber the local build. You may want to try clobbering as well?

Oh, wait, this issue: Issue #29082: Fixed loading libraries in ctypes by unicode names on Windows appears to be a regression in Python 2.7.13 that is fixed in Python 2.7.14 https://bugs.python.org/issue29082

I have clobbered many many times so I'm pretty sure that's not it. It's not even related to the mozilla-central checkout. I can reproduce this just by running `c:/mozilla-build/python/Scripts/hg.exe --version` from anywhere.

(In reply to Brian Birtles (:birtles) from comment #5) > Oh, wait, this issue: > > Issue #29082: Fixed loading libraries in ctypes by unicode names on Windows > > appears to be a regression in Python 2.7.13 that is fixed in Python 2.7.14 (Which is odd, because I see this bug when using 2.7.14 but *not* when using 2.7.13. Maybe this is not related after all.)

I can confirm this issue without any Japanese locale. I encounter this, even when just running "mach bootstrap": Successfully installed Mercurial-4.4.1 Your version of Python (2.7.14) is new enough. Your version of Rust (1.21.0) is new enough. Rust supports i686-pc-windows-msvc, x86_64-pc-windows-msvc targets. Traceback (most recent call last): File "c:\mozilla-build\python\Scripts\hg", line 41, in <module> dispatch.run() File "c:\mozilla-build\python\lib\site-packages\hgdemandimport\demandimportpy2.py", line 145, in __getattr__ self._load() File "c:\mozilla-build\python\lib\site-packages\hgdemandimport\demandimportpy2.py", line 90, in _load mod = _hgextimport(_origimport, head, globals, locals, None, level) File "c:\mozilla-build\python\lib\site-packages\hgdemandimport\demandimportpy2.py", line 41, in _hgextimport return importfunc(name, globals, *args, **kwargs) File "c:\mozilla-build\python\lib\site-packages\mercurial\dispatch.py", line 536, in <module> class lazyaliasentry(object): File "c:\mozilla-build\python\lib\site-packages\mercurial\dispatch.py", line 545, in lazyaliasentry @util.propertycache File "c:\mozilla-build\python\lib\site-packages\hgdemandimport\demandimportpy2.py", line 145, in __getattr__ self._load() File "c:\mozilla-build\python\lib\site-packages\hgdemandimport\demandimportpy2.py", line 90, in _load mod = _hgextimport(_origimport, head, globals, locals, None, level) File "c:\mozilla-build\python\lib\site-packages\hgdemandimport\demandimportpy2.py", line 41, in _hgextimport return importfunc(name, globals, *args, **kwargs) File "c:\mozilla-build\python\lib\site-packages\mercurial\util.py", line 97, in <module> stdout = platform.winstdout(stdout) File "c:\mozilla-build\python\lib\site-packages\hgdemandimport\demandimportpy2.py", line 145, in __getattr__ self._load() File "c:\mozilla-build\python\lib\site-packages\hgdemandimport\demandimportpy2.py", line 90, in _load mod = _hgextimport(_origimport, head, globals, locals, None, level) File "c:\mozilla-build\python\lib\site-packages\hgdemandimport\demandimportpy2.py", line 41, in _hgextimport return importfunc(name, globals, *args, **kwargs) File "c:\mozilla-build\python\lib\site-packages\mercurial\windows.py", line 34, in <module> executablepath = win32.executablepath File "c:\mozilla-build\python\lib\site-packages\hgdemandimport\demandimportpy2.py", line 145, in __getattr__ self._load() File "c:\mozilla-build\python\lib\site-packages\hgdemandimport\demandimportpy2.py", line 90, in _load mod = _hgextimport(_origimport, head, globals, locals, None, level) File "c:\mozilla-build\python\lib\site-packages\hgdemandimport\demandimportpy2.py", line 41, in _hgextimport return importfunc(name, globals, *args, **kwargs) File "c:\mozilla-build\python\lib\site-packages\mercurial\win32.py", line 22, in <module> _kernel32 = ctypes.windll.kernel32 File "c:\mozilla-build\python\lib\site-packages\hgdemandimport\demandimportpy2.py", line 145, in __getattr__ self._load() File "c:\mozilla-build\python\lib\site-packages\hgdemandimport\demandimportpy2.py", line 90, in _load mod = _hgextimport(_origimport, head, globals, locals, None, level) File "c:\mozilla-build\python\lib\site-packages\hgdemandimport\demandimportpy2.py", line 41, in _hgextimport return importfunc(name, globals, *args, **kwargs) File "c:\mozilla-build\python\lib\ctypes\__init__.py", line 7, in <module> from _ctypes import Union, Structure, Array File "c:\mozilla-build\python\lib\site-packages\hgdemandimport\demandimportpy2.py", line 268, in _demandimport mod = _hgextimport(_origimport, name, globals, locals) File "c:\mozilla-build\python\lib\site-packages\hgdemandimport\demandimportpy2.py", line 41, in _hgextimport return importfunc(name, globals, *args, **kwargs) ImportError: DLL load failed: The specified procedure could not be found.

There seems to be something funny with _ctypes.pyd. If I change ctypes/__init__.py to make the following line the first of the _ctypes imports: from _ctypes import __version__ as _ctypes_version it errors out there saying it can't find the specified procedure.

If I copy C:\mozilla-build\python\DLLs\_ctypes.pyd from a MozillaBuild 3.0 install to my MozillaBuild 3.1 install it works.

Those DLLs have the same version (1.1.0) but the files are clearly different. I guess they were compiled with different flags/compiler and whatever that difference was, it seems to be causing problems here. I guess it's possible that other Python DLLs are similarly affected but so far my build seems to be running.

_ctypes.pyd from MozillaBuild 3.1 has a dependency on MSVCR90.DLL. This file won't be present on fresh Windows installs. This version of the MSVCRT will get installed automatically with a ton of software. So if you "bootstrap" a fresh Windows install beyond MozillaBuild, something will likely install it. To work around this failure, do a search for "Visual Studio 2008 Redistributable" then download and install it from Microsoft. Also, if you try to launch an executable that requires this missing file, my recollection is Windows will automagically open a dialog box asking if you want to install the missing dependency. The proper fix is for the MozillaBuild installer to install this dependency automatically. Or it shouldn't install software relying on DLLs that aren't installed/managed by MozillaBuild. I wouldn't be surprised if the installer software we are using for MozillaBuild has an option to automatically include the Visual Studio 2008 Redistributable. That's a pretty common dependency and I've seen lots of installers pull in this dependency automatically in fresh VMs.

And to be clear, the culprit is likely the Python 2.7 package. Mercurial is just likely the first thing to load the ctypes Python package to uncover it.

The _ctypes.pyd from MozillaBuild 2.2 and 3.0 also had dependencies on msvcr90.dll. However, MozillaBuild 2.2's Python was 32-bit. There are 32 and 64 bit versions of msvcr90.dll. It is possible to have only one installed.

Running the downloaded vcredist_x86.exe just gives me "Repair" and "Uninstall" options. After running "Repair" I don't see any additional copies of msvcr90.dll on my system (there were already 27 copies). It also doesn't seem to fix the problem. Likewise downloading and running vcredist_x64.exe just presents "Repair" and "Uninstall" and doesn't appear to fix the problem.

This is not a particularly fresh install of Windows, for what it's worth. I've used the machine for about 7~8 months. Also MozillaBuild 3.0 works fine and it uses 64-bit Python.

With the build in comment 17, I still get the same error as comment 8: File "c:\mozilla-build\python\lib\site-packages\hgdemandimport\demandimportpy2.py", line 41, in _hgextimport return importfunc(name, globals, *args, **kwargs) ImportError: DLL load failed: The specified procedure could not be found. (I had moved my old mozilla-build folder out of the way) I additionally confirm that MozillaBuild 3.0 continues to work well for me.

(In reply to Brian Birtles (:birtles) from comment #11) > Those DLLs have the same version (1.1.0) but the files are clearly > different. I guess they were compiled with different flags/compiler and > whatever that difference was, it seems to be causing problems here. I looked at the _ctypes.pyd from MozillaBuild 3.0, it has SHA-256 hash: B21637CE6ACD18DFE911A0392F491DA9DCA3787F66FB8AD0B50EABC2EC37C1F9 whereas the ones for MozillaBuild 3.1 and :ryanvm's test build in comment 17 are identical - they have the following SHA-256 hash: 14DF3E276E4AB33AEC0731201DD6CD571BAFF1B7F21430CC1E4E0D36251475BE

I ran both _ctypes.pyd files through `dumpbin.exe /all` and diffed the results and didn't see anything obvious.

(Now I get `abort: cannot import name _remove_dead_weakref` when running './mach build' from *either* MozillaBuild 3.0 or MozillaBuild 3.1 although it seems like MozillaBuild 3.0 is somehow picking up the 3.1 version of hg. I've just switched back to MozillaBuild 3.0 wholesale instead of monkey-patching MozillaBuild 3.1 and I've also removed MozillaBuild 3.1 entirely and now building works fine. I've had a bad run this week with bug 1414060, then bug 1415364, then this bug, then bug 1415802 so I'll hold off on trying MozillaBuild 3.1 for few more weeks.)

I can reproduce this on my home desktop in my primary OS environment. hg.exe demonstrates the problem. The hg script does not. I can reproduce with the Mercurial in MozillaBuild and the Mercurial from PyPI. I can reproduce with Mercurial 4.4, 4.3.2, and 4.2.3 wheels from PyPI. I also get the failure if I compile Mercurial 4.4.1 from source in MozillaBuild 3.1. I reckon this has something to do with the Python upgrade in MozillaBuild as opposed to a change in Mercurial. It potentially has something to do with packaging. Perhaps hg.exe is picking up the wrong Python shared library or something. Here's how I tested: $ HGDEMANDIMPORT=disable /c/dev/mozilla-build/python/Scripts/hg.exe Traceback (most recent call last): File "c:\dev\mozilla-build\python\Scripts\hg", line 40, in <module> from mercurial import dispatch File "c:\dev\mozilla-build\python\Lib\site-packages\mercurial\dispatch.py", line 22, in <module> from .i18n import _ File "c:\dev\mozilla-build\python\Lib\site-packages\mercurial\i18n.py", line 10, in <module> import gettext as gettextmod File "c:\dev\mozilla-build\python\lib\gettext.py", line 49, in <module> import locale, copy, os, re, struct, sys File "c:\dev\mozilla-build\python\lib\copy.py", line 52, in <module> import weakref File "c:\dev\mozilla-build\python\lib\weakref.py", line 14, in <module> from _weakref import ( ImportError: cannot import name _remove_dead_weakref

The error about _remove_dead_weakref is interesting because this symbol was added in Python 2.7.14. When I trace the execution of hg.exe in MozillaBuild 3.1, I see that it picks up python27.dll from c:\Windows\System32\python27.dll. (It first attempts .\python27.dll.) It subsequently loads .py and .pyc modules from the MozillaBuild directory. So essentially we're getting a franken-python where the main Python interpreter is <2.7.14 and the .py files are 2.7.14. Since the 2.7.14 .py files attempt to reference a symbol defined in C that isn't present in the <2.7.14 DLL, Python errors. If I trace hg.exe in MozillaBuild 3.0, I get the exact same behavior. However, since my c:\Windows\System32\python27.dll is from a new enough Python (possibly 2.7.13), everything "just works." So, this is definitely a bug in the way hg.exe is loading python27.dll. Whether it is a bug in MozillaBuild or a bug in hg.exe, I'm not sure. hg.exe is basically doing a LoadLibrary("python27.dll") here. According to https://msdn.microsoft.com/en-us/library/windows/desktop/ms682586(v=vs.85).aspx#search_order_for_desktop_applications the system path (e.g. c:\Windows\System32) is consulted pretty early on. So since there isn't a python27.dll in MozillaBuild's python\Scripts directory... One workaround for this is to copy hg.exe and hg from python/Scripts/ to python/. Another is to copy python/python27.dll to python/Scripts/. Either way, hg.exe and python27.dll are in the same directory and the default DLL load rules mean that DLL is loaded. Copying python27.dll into Scripts/ is more robust because if pip installs a new Mercurial, it will put hg.exe into Scripts/. We don't want an orphan copy sitting around in PATH in python/. But I suspect there is a bug in Mercurial's packaging here. I wouldn't be surprised if Mercurial is supposed to be using some different mechanism for producing the hg.exe so that it knows exactly which python shared library to load. That being said, my standalone Python 2.7 install doesn't have a python27.dll. Yet my standalone Python 3.6 install does have a python36.dll. I'm not really sure what's going on in Python DLL management land. Alex: do you know anything about this aspect of Python packaging?

Mid-air collision dropped my needinfo...

Ahh - if you run the Python Windows MSI, it asks to install for "all users" or "just for me." If you select "all users," it puts python27.dll in c:\Windows\System32. If "just for me," it goes under %INSTALL%\python27.dll. So that's likely where my system python27.dll came from. Still doesn't solve the problem of hg.exe.

Can someone else confirm that copying python27.dll into the Scripts directory makes this problem go away? I'm fully on board with releasing a 3.1.1 hotfix for this issue if that's sufficient to work around it.

Unfortunately I don't know much about linkage on Windows.

I verify that copying python27.dll from the python/ directory to python/Scripts makes the problem go away for me, so `mach bootstrap` works as expected after that.

Re-installed MozillaBuild 3.1 alongside 3.0 and ran `./mach clobber`. Confirmed that `./mach build` doesn't work. Copied python.dll from python/ to python/Scripts. Confirmed that `./mach build` now works.

It seems (due to still building) that I can build with copying the dll (i.e., same as Brian).

Just FTR, I recently updated mozilla-build on my Windows machine (I forget exactly what prompted me to do this -- I was having some kind of build issue, and figured updating to the latest tools was usually a good idea), and then spent entirely too much time struggling with a completely broken system, before I came here and found that it's known.... the failures I was hitting looked just like those reported above by Brian. (Copying python.dll from python/ to python/Scripts did not fix things for me, but it's possible that I had messed up something else by the time I got here and tried that. I ended up reverting to a fresh mozilla-build 3.0.)

I can confirm. I was facing the same issue when rebuilding. However, copying the `python27.dll` to the `Scripts/` directory fixed the issue for me. Trace when I faced this error: https://pastebin.mozilla.org/9073293

I've uploaded a test build of 3.1.1 includes the DLL workaround over in bug 1418484 comment 1. Would love to get some confirmation from affected users that everything is copacetic with it.

I can run: hg --version and: ./mach bootstrap successfully with the test build of 3.1.1 that :ryanvm uploaded.

I installed MozillaBuild 3.1 today and promptly hit this problem. I think gps' comment 24 and comment 26 are essentially the root cause. I was not aware that I had Python 2.7 installed system-wide! Uninstalling it (from Add/Remove Programs) made hg.exe find the right python27.dll (due to it being in $PATH in the MozillaBuild shell).

(In reply to Gary Kwong [:gkw] [:nth10sd] from comment #39) > I can run: > > hg --version > > and: > > ./mach bootstrap > > successfully with the test build of 3.1.1 that :ryanvm uploaded. Works for me too.

Pushed by ryanvm@gmail.com: https://hg.mozilla.org/mozilla-build/rev/5976d55cf6e9 Copy python27.dll to the python\Scripts directory to work around path detection issues in hg.exe.