There are error and warning messages in the build log and the old-configure is stopped. DEBUG: cl : Command line warning D9035 : option 'o' has been deprecated and will be removed in a future release DEBUG: LINK : fatal error LNK1104: cannot open file 'LIBCMT.lib' From https://archive.mozilla.org/pub/thunderbird/nightly/2016/04/2016-04-24-03-02-56-comm-central-l10n/comm-central-win32-l10n-nightly-ja-bm91-build1-build267.txt.gz ========= Started make config failed (results: 2, elapsed: 42 secs) (at 2016-04-24 04:41:28.701310) ========= ... checking whether the C compiler (c:/builds/moz2_slave/tb-c-cen-w32-l10n-ntly-0000000/build/comm-central/vs2015u1/VC/bin/amd64_x86/cl.EXE ) works... no configure: error: installation or configuration problem: C compiler cannot create executables. DEBUG: <truncated - see config.log for full output> DEBUG: configure:2802: c:/builds/moz2_slave/tb-c-cen-w32-l10n-ntly-0000000/build/comm-central/vs2015u1/VC/bin/amd64_x86/cl.EXE -o conftest conftest.c 1>&5 DEBUG: Microsoft (R) C/C++ Optimizing Compiler Version 19.00.23506 for x86 DEBUG: Copyright (C) Microsoft Corporation. All rights reserved. DEBUG: DEBUG: cl : Command line warning D9035 : option 'o' has been deprecated and will be removed in a future release DEBUG: conftest.c DEBUG: Microsoft (R) Incremental Linker Version 14.00.23506.0 DEBUG: Copyright (C) Microsoft Corporation. All rights reserved. DEBUG: DEBUG: /out:conftest.exe DEBUG: /out:conftest.exe DEBUG: conftest.obj DEBUG: LINK : fatal error LNK1104: cannot open file 'LIBCMT.lib' DEBUG: configure: failed program was: DEBUG: DEBUG: #line 2797 "configure" DEBUG: #include "confdefs.h" DEBUG: DEBUG: main(){return(0);} DEBUG: configure: error: installation or configuration problem: C compiler cannot create executables. ERROR: old-configure failed

Probably the mail l10n-mozconfigs are out of date.

I'm not sure why the mail/ l10n-mozconfigs are so stripped-down compared to the regular nightly mozconfig, and/or why they don't include some common mozconfig like browser/. What's the logic here for what gets included where?

IIRC the l10n mozconfigs don't need the full configuration because it is used only for repacks. That said, the differences between the files don't look like they could change anything w.r.t. the error message. This may be fallout from releng upgrading to vs2015 maybe, the internet suggests that paths are not set up correctly with this error message.

(In reply to Philipp Kewisch [:Fallen] from comment #3) > IIRC the l10n mozconfigs don't need the full configuration because it is > used only for repacks. That said, the differences between the files don't > look like they could change anything w.r.t. the error message. This may be > fallout from releng upgrading to vs2015 maybe, the internet suggests that > paths are not set up correctly with this error message. Or some fallout from the configure changes. My guess about the mozconfigs was because e.g. $topsrcdir in mozconfigs no longer points where it used to, which can cause path issues. Fwiw on OSX the failure is different: checking for the target C++ compiler... /usr/bin/clang++ checking whether the target C++ compiler can be used... DEBUG: <truncated - see config.log for full output> DEBUG: | %COMPILER clang DEBUG: | # if !__cplusplus || __cpp_static_assert DEBUG: | %VERSION __clang_major__.__clang_minor__.__clang_patchlevel__ DEBUG: | # endif DEBUG: | #elif defined(__GNUC__) DEBUG: | %COMPILER gcc DEBUG: | %VERSION __GNUC__.__GNUC_MINOR__.__GNUC_PATCHLEVEL__ DEBUG: | #endif DEBUG: | DEBUG: | #if __cplusplus DEBUG: | %cplusplus __cplusplus DEBUG: | #elif __STDC_VERSION__ DEBUG: | %STDC_VERSION __STDC_VERSION__ DEBUG: | #elif __STDC__ DEBUG: | %STDC_VERSION 198900L DEBUG: | #endif DEBUG: Executing: `/usr/bin/clang++ -std=gnu++11 -E /var/folders/0l/_g5gq80d4zq46sd3t9xrz1v400000w/T/conftest.UPT9Ca.cpp` DEBUG: The command returned non-zero exit status 1. DEBUG: Its error output was: DEBUG: | error: invalid value 'gnu++11' in '-std=gnu++11' ERROR: Command `/usr/bin/clang++ -std=gnu++11 -E /var/folders/0l/_g5gq80d4zq46sd3t9xrz1v400000w/T/conftest.UPT9Ca.cpp` failed with exit status 1.

Reproduced on the latest comm-aurora-l10n (49.0a2).

This is impacting THunderbird Aurora for many months (see bug 1222878). Can you offer advice to get us out of this long funk? Is more information needed?

https://hg.mozilla.org/releases/comm-aurora/rev/b6fe48de6b8cf111d140e0e24918210ec72d6e8e Experimental l10n-mozconfig change to investigate Bug 1267523. a=aleth

> DEBUG: LINK : fatal error LNK1104: cannot open file 'LIBCMT.lib' suggests bad paths. But why aren't you doing l10n repacks with --disable-compile-environment?

https://hg.mozilla.org/releases/comm-aurora/rev/0240547b3efde85daa8de4d08216626409580d0d Backed out changeset b6fe48de6b8c (Bug 1267523). a=aleth

If we need more users and testers, l10n build would help it. Are there no needs for comm-central-l10n and comm-aurora-l10n builds?

(In reply to Takanori MATSUURA from comment #10) > If we need more users and testers, l10n build would help it. > Are there no needs for comm-central-l10n and comm-aurora-l10n builds? The problem with fixing this is how to test potential patches. Do you know how to do comm-central-l10n builds on try?

(In reply to aleth [:aleth] from comment #11) > The problem with fixing this is how to test potential patches. Do you know > how to do comm-central-l10n builds on try? Sorry. I don't know how to do it.

https://hg.mozilla.org/releases/comm-aurora/rev/1b5ca34bd56e48f811c5b1bb341862f1041db587 Experimental l10n-mozconfig change to investigate Bug 1267523. a=aleth

Looks like we currently don't even get to configure :-(

Note for the future: Bug 1286440 might be related.

Following Bug 1297786 and comment 8, we should try adding --disable-compile-environment to all the l10n mozconfigs that don't have it yet. If that doesn't work, we'll have to look at getting tooltool to work similar to what was done for beta release.

Comment on attachment 8787915 [details] [diff] [review] Add disable-compile-environment to l10n mozconfigs lgtm, but this is in mail/ and I can't technically review it. I think either jcranmer, or mkmelin (I think) would be the right person.

Comment on attachment 8787915 [details] [diff] [review] Add disable-compile-environment to l10n mozconfigs Review of attachment 8787915 [details] [diff] [review]: ----------------------------------------------------------------- I'm not in a position to test this, but the risk is minimal and experienced people have said this is a good idea. So if you need a review, you can rs=me

https://hg.mozilla.org/comm-central/rev/e472357807bbf83554d0060003e22cb34056e849 Bug 1267523 - Add disable-compile-environment to l10n mozconfigs. r=ewong rs=rkent

Doesn't seem to have helped on its own, on Linux it's still looking for PKG_CONFIG, because mozconfig.gtk is included. Adding configure options from /builds/slave/tb-c-cen-l64-l10n-ntly-0000000/build/comm-central/.mozconfig --enable-crashreporter --enable-release --enable-js-shell --enable-elf-hack --enable-stdcxx-compat --enable-default-toolkit=cairo-gtk3 --enable-application=mail --enable-update-channel=nightly --with-l10n-base=../../l10n --disable-compile-environment --enable-calendar --disable-stdcxx-compat PKG_CONFIG_SYSROOT_DIR=/builds/slave/tb-c-cen-l64-l10n-ntly-0000000/build/comm-central/mozilla/gtk3 PKG_CONFIG_LIBDIR=/usr/lib64/pkgconfig:/usr/share/pkgconfig SOCORRO_SYMBOL_UPLOAD_TOKEN_FILE=/builds/crash-stats-api.token MOZILLA_OFFICIAL=1 PKG_CONFIG_PATH=/builds/slave/tb-c-cen-l64-l10n-ntly-0000000/build/comm-central/mozilla/gtk3/usr/local/lib/pkgconfig MOZ_DEBUG_SYMBOLS=1 PATH=/builds/slave/tb-c-cen-l64-l10n-ntly-0000000/build/comm-central/mozilla/gtk3/usr/local/bin:/builds/slave/tb-c-cen-l64-l10n-ntly-0000000/build/comm-central/gcc/bin:/builds/slave/tb-c-cen-l64-l10n-ntly-0000000/build/comm-central/gcc/bin:/tools/buildbot/bin:/usr/local/bin:/usr/lib64/ccache:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/tools/git/bin:/tools/python27/bin:/tools/python27-mercurial/bin:/home/cltbld/bin LD_LIBRARY_PATH=/tools/gcc-4.3.3/installed/lib64:/builds/slave/tb-c-cen-l64-l10n-ntly-0000000/build/comm-central/gcc/lib64:/builds/slave/tb-c-cen-l64-l10n-ntly-0000000/build/comm-central/gcc/lib:/builds/slave/tb-c-cen-l64-l10n-ntly-0000000/build/comm-central/gtk3/usr/local/lib:/builds/slave/tb-c-cen-l64-l10n-ntly-0000000/build/comm-central/mozilla/gtk3/usr/local/lib MOZ_ADDON_SIGNING=0 MOZ_REQUIRE_SIGNING=0 CC=/builds/slave/tb-c-cen-l64-l10n-ntly-0000000/build/comm-central/mozilla/gcc/bin/gcc LDFLAGS=-L/builds/slave/tb-c-cen-l64-l10n-ntly-0000000/build/comm-central/mozilla/gtk3/usr/local/lib TOOLTOOL_DIR=/builds/slave/tb-c-cen-l64-l10n-ntly-0000000/build/comm-central/mozilla MOZ_REQUIRE_ADDON_SIGNING=0 CXX=/builds/slave/tb-c-cen-l64-l10n-ntly-0000000/build/comm-central/mozilla/gcc/bin/g++ PKG_CONFIG=/builds/slave/tb-c-cen-l64-l10n-ntly-0000000/build/comm-central/mozilla/gtk3/usr/local/bin/pkg-config checking for a shell... /bin/sh checking for host system type... x86_64-pc-linux-gnu checking for target system type... x86_64-pc-linux-gnu checking whether cross compiling... no checking for pkg_config... not found DEBUG: pkg_config: Trying /builds/slave/tb-c-cen-l64-l10n-ntly-0000000/build/comm-central/mozilla/gtk3/usr/local/bin/pkg-config ERROR: Cannot find pkg_config On Windows we have https://ftp.mozilla.org/pub/thunderbird/nightly/2016/09/2016-09-07-17-49-56-comm-central-l10n/comm-central-win32-l10n-nightly-ga-IE-bm91-build1-build149.txt.gz Executing command: ['python', '../tools/buildfarm/utils/hgtool.py', '-r', 'default', 'https://hg.mozilla.org/mozilla-central', 'c:/builds/moz2_slave/tb-c-cen-w32-l10n-ntly-0000000/build/comm-central\\mozilla'] TinderboxPrint:<a href=https://hg.mozilla.org/mozilla-central/rev/33e7ae9b3104 title='Built from mozilla-central revision 33e7ae9b3104'>moz:33e7ae9b3104</a> c:/builds/moz2_slave/tb-c-cen-w32-l10n-ntly-0000000/build/comm-central/mozmake.exe -f c:/builds/moz2_slave/tb-c-cen-w32-l10n-ntly-0000000/build/comm-central/client.mk configure mozmake.exe[1]: Entering directory 'c:/builds/moz2_slave/tb-c-cen-w32-l10n-ntly-0000000/build/comm-central' cd c:/builds/moz2_slave/tb-c-cen-w32-l10n-ntly-0000000/build/comm-central/objdir-tb c:/builds/moz2_slave/tb-c-cen-w32-l10n-ntly-0000000/build/comm-central/configure Traceback (most recent call last): File "c:/builds/moz2_slave/tb-c-cen-w32-l10n-ntly-0000000/build/comm-central/configure.py", line 32, in <module> sys.exit(main(sys.argv)) File "c:/builds/moz2_slave/tb-c-cen-w32-l10n-ntly-0000000/build/comm-central/configure.py", line 24, in main sandbox.run(os.path.join(os.path.dirname(__file__), 'moz.configure')) File "c:\builds\moz2_slave\tb-c-cen-w32-l10n-ntly-0000000\build\comm-central\mozilla\python\mozbuild\mozbuild\configure\__init__.py", line 231, in run self.include_file(path) File "c:\builds\moz2_slave\tb-c-cen-w32-l10n-ntly-0000000\build\comm-central\mozilla\python\mozbuild\mozbuild\configure\__init__.py", line 222, in include_file exec_(code, self) File "c:\builds\moz2_slave\tb-c-cen-w32-l10n-ntly-0000000\build\comm-central\mozilla\python\mozbuild\mozbuild\util.py", line 59, in exec_ exec(object, globals, locals) File "c:/builds/moz2_slave/tb-c-cen-w32-l10n-ntly-0000000/build/comm-central/moz.configure", line 7, in <module> include('mozilla/moz.configure') File "c:\builds\moz2_slave\tb-c-cen-w32-l10n-ntly-0000000\build\comm-central\mozilla\python\mozbuild\mozbuild\configure\__init__.py", line 477, in include_impl self.include_file(what) File "c:\builds\moz2_slave\tb-c-cen-w32-l10n-ntly-0000000\build\comm-central\mozilla\python\mozbuild\mozbuild\configure\__init__.py", line 222, in include_file exec_(code, self) File "c:\builds\moz2_slave\tb-c-cen-w32-l10n-ntly-0000000\build\comm-central\mozilla\python\mozbuild\mozbuild\util.py", line 59, in exec_ exec(object, globals, locals) File "c:/builds/moz2_slave/tb-c-cen-w32-l10n-ntly-0000000/build/comm-central/mozilla/moz.configure", line 7, in <module> include('build/moz.configure/init.configure') File "c:\builds\moz2_slave\tb-c-cen-w32-l10n-ntly-0000000\build\comm-central\mozilla\python\mozbuild\mozbuild\configure\__init__.py", line 477, in include_impl self.include_file(what) File "c:\builds\moz2_slave\tb-c-cen-w32-l10n-ntly-0000000\build\comm-central\mozilla\python\mozbuild\mozbuild\configure\__init__.py", line 222, in include_file exec_(code, self) File "c:\builds\moz2_slave\tb-c-cen-w32-l10n-ntly-0000000\build\comm-central\mozilla\python\mozbuild\mozbuild\util.py", line 59, in exec_ exec(object, globals, locals) File "c:/builds/moz2_slave/tb-c-cen-w32-l10n-ntly-0000000/build/comm-central/mozilla/build/moz.configure/init.configure", line 90, in <module> @imports(_from='mozbuild.mozconfig', _import='MozconfigLoader') File "c:\builds\moz2_slave\tb-c-cen-w32-l10n-ntly-0000000\build\comm-central\mozilla\python\mozbuild\mozbuild\configure\__init__.py", line 458, in decorator self._value_for(dummy) File "c:\builds\moz2_slave\tb-c-cen-w32-l10n-ntly-0000000\build\comm-central\mozilla\python\mozbuild\mozbuild\configure\__init__.py", line 302, in _value_for return self._value_for_depends(obj) File "c:\builds\moz2_slave\tb-c-cen-w32-l10n-ntly-0000000\build\comm-central\mozilla\python\mozbuild\mozbuild\util.py", line 924, in method_call cache[args] = self.func(instance, *args) File "c:\builds\moz2_slave\tb-c-cen-w32-l10n-ntly-0000000\build\comm-central\mozilla\python\mozbuild\mozbuild\configure\__init__.py", line 329, in _value_for_depends return func(*resolved_args) File "c:\builds\moz2_slave\tb-c-cen-w32-l10n-ntly-0000000\build\comm-central\mozilla\python\mozbuild\mozbuild\configure\__init__.py", line 773, in wrapped return new_func(*args, **kwargs) File "c:/builds/moz2_slave/tb-c-cen-w32-l10n-ntly-0000000/build/comm-central/mozilla/build/moz.configure/init.configure", line 118, in mozconfig mozconfig = loader.read_mozconfig(mozconfig, moz_build_app=current_project) File "c:\builds\moz2_slave\tb-c-cen-w32-l10n-ntly-0000000\build\comm-central\mozilla\python\mozbuild\mozbuild\mozconfig.py", line 255, in read_mozconfig raise MozconfigLoadException(path, MOZCONFIG_BAD_EXIT_CODE, lines) mozbuild.mozconfig.MozconfigLoadException: Evaluation of your mozconfig exited with an error. This could be triggered by a command inside your mozconfig failing. Please change your mozconfig to not error and/or to catch errors in executed commands. *** Fix above errors and then restart with\ "c:/builds/moz2_slave/tb-c-cen-w32-l10n-ntly-0000000/build/comm-central/mozmake.exe -f client.mk build" c:/builds/moz2_slave/tb-c-cen-w32-l10n-ntly-0000000/build/comm-central/client.mk:358: recipe for target 'configure' failed mozmake.exe[1]: Leaving directory 'c:/builds/moz2_slave/tb-c-cen-w32-l10n-ntly-0000000/build/comm-central' mozmake.exe[1]: *** [configure] Error 1 client.mk:215: recipe for target 'configure' failed mozmake.exe: *** [configure] Error 2

On OSX we get a bit further but the checkout fails with merge conflicts (?!) https://ftp.mozilla.org/pub/thunderbird/nightly/2016/09/2016-09-07-17-49-57-comm-central-l10n/comm-central-macosx64-l10n-nightly-be-bm82-build1-build2082.txt.gz python client.py checkout --mozilla-repo=https://hg.mozilla.org/mozilla-central --skip-inspector --skip-venkman --skip-chatzilla in dir /builds/slave/tb-c-cen-m64-l10n-ntly-0000000/build/comm-central (timeout 3600 secs) watching logfiles {} argv: ['python', 'client.py', 'checkout', '--mozilla-repo=https://hg.mozilla.org/mozilla-central', '--skip-inspector', '--skip-venkman', '--skip-chatzilla'] environment: Apple_PubSub_Socket_Render=/tmp/launch-d1P6Oh/Render DISPLAY=/tmp/launch-1OeuW8/org.x:0 GIT_SHARE_BASE_DIR=/builds/git-shared HG_SHARE_BASE_DIR=/builds/hg-shared HOME=/Users/cltbld IDLEIZER_DISABLE_SHUTDOWN=true LOGNAME=cltbld PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/bin/X11 PWD=/builds/slave/tb-c-cen-m64-l10n-ntly-0000000/build/comm-central RUNNER_CONFIG_CMD=/opt/runner/bin/python2.7 /opt/runner/bin/runner -c /opt/runner/runner.cfg SHELL=/bin/bash SSH_AUTH_SOCK=/tmp/launch-rzDzMr/Listeners TMPDIR=/var/folders/z_/5wdr30sn0rb458pryz5zmylr00000w/T/ TWISTD_LOG_PATH=/builds/slave/twistd.log USER=cltbld VERSIONER_PYTHON_PREFER_32_BIT=no VERSIONER_PYTHON_VERSION=2.7 __CF_USER_TEXT_ENCODING=0x1C:0:0 using PTY: False pulling from https://hg.mozilla.org/comm-central searching for changes no changes found 0 files updated, 0 files merged, 0 files removed, 0 files unresolved Updated to revision 37f7bb383f98baadaf6d3e8ccf36608f46e0dc90. pulling from https://hg.mozilla.org/mozilla-central searching for changes no changes found Traceback (most recent call last): File "/tools/python27-mercurial/lib/python2.7/site-packages/mercurial/dispatch.py", line 191, in _runcatch return _dispatch(req) File "/tools/python27-mercurial/lib/python2.7/site-packages/mercurial/dispatch.py", line 924, in _dispatch cmdpats, cmdoptions) File "/tools/python27-mercurial/lib/python2.7/site-packages/mercurial/dispatch.py", line 681, in runcommand ret = _runcommand(ui, options, cmd, d) File "/tools/python27-mercurial/lib/python2.7/site-packages/mercurial/dispatch.py", line 1055, in _runcommand return checkargs() File "/tools/python27-mercurial/lib/python2.7/site-packages/mercurial/dispatch.py", line 1015, in checkargs return cmdfunc() File "/tools/python27-mercurial/lib/python2.7/site-packages/mercurial/dispatch.py", line 921, in <lambda> d = lambda: util.checksignature(func)(ui, *args, **cmdoptions) File "/tools/python27-mercurial/lib/python2.7/site-packages/mercurial/util.py", line 991, in check return func(*args, **kwargs) File "/tools/python27-mercurial/lib/python2.7/site-packages/mercurial/extensions.py", line 195, in closure return func(*(args + a), **kw) File "/tools/python27-mercurial/lib/python2.7/site-packages/mercurial/util.py", line 991, in check return func(*args, **kwargs) File "/tools/python27-mercurial/lib/python2.7/site-packages/hgext/mq.py", line 3517, in mqcommand return orig(ui, repo, *args, **kwargs) File "/tools/python27-mercurial/lib/python2.7/site-packages/mercurial/util.py", line 991, in check return func(*args, **kwargs) File "/tools/python27-mercurial/lib/python2.7/site-packages/mercurial/commands.py", line 6973, in update ret = hg.update(repo, rev) File "/tools/python27-mercurial/lib/python2.7/site-packages/mercurial/hg.py", line 680, in update stats = updaterepo(repo, node, False) File "/tools/python27-mercurial/lib/python2.7/site-packages/mercurial/hg.py", line 676, in updaterepo labels=['working copy', 'destination']) File "/tools/python27-mercurial/lib/python2.7/site-packages/mercurial/merge.py", line 1403, in update raise error.Abort(_("outstanding merge conflicts")) Abort: outstanding merge conflicts abort: outstanding merge conflicts Executing command: ['hg', 'pull', '-R', './.'] Executing command: ['hg', 'update', '-r', 'default', '-R', './.'] Executing command: ['hg', 'pull', '-R', './mozilla', 'https://hg.mozilla.org/mozilla-central'] Executing command: ['hg', 'update', '-r', 'default', '-R', './mozilla'] Traceback (most recent call last): File "client.py", line 665, in <module> options.mozilla_rev, options.hgtool, options.hgtool1) File "client.py", line 380, in do_hg_pull check_call_noisy(cmd, retryMax=0) File "client.py", line 139, in check_call_noisy execute_check_call(cmd, *args, **kwargs) File "client.py", line 135, in execute_check_call check_call(cmd, *args, **kwargs) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 504, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['hg', 'update', '-r', 'default', '-R', './mozilla']' returned non-zero exit status 255 program finished with exit code 1

All fixes in the dependent bugs.