Failed a release build because it couldn't upload symbols: (view as text) 'python' 'e:/builds/moz2_slave/rel-m-rel-w32_bld-000000000000/tools/buildfarm/utils/retry.py' '-s' '1' '-r' '5' '-t' '2460' 'python' 'e:/builds/moz2_slave/rel-m-rel-w32_bld-000000000000/build/build/pymake/make.py' 'uploadsymbols' in dir e:\builds\moz2_slave\rel-m-rel-w32_bld-000000000000\build/obj-firefox (timeout 2520 secs) watching logfiles {} argv: ['python', 'e:/builds/moz2_slave/rel-m-rel-w32_bld-000000000000/tools/buildfarm/utils/retry.py', '-s', '1', '-r', '5', '-t', '2460', 'python', 'e:/builds/moz2_slave/rel-m-rel-w32_bld-000000000000/build/build/pymake/make.py', 'uploadsymbols'] environment: !EXITCODE=00000001 ALLUSERSPROFILE=C:\ProgramData APPDATA=C:\Users\cltbld\AppData\Roaming APR_ICONV_PATH=c:/mozilla-build/svn-win32-1.6.3/iconv BINSCOPE=C:\Program Files (x86)\Microsoft\SDL BinScope\BinScope.exe BUILDBOTCUSTOM=http://hg.mozilla.org/build/buildbotcustom BUILDBOT_CONFIGS=http://hg.mozilla.org/build/buildbot-configs CLOBBERER_URL=http://clobberer.pvt.build.mozilla.org/always_clobber.php COMMONPROGRAMFILES=C:\Program Files (x86)\Common Files COMMONPROGRAMFILES(X86)=C:\Program Files (x86)\Common Files COMMONPROGRAMW6432=C:\Program Files\Common Files COMM_REV=FIREFOX_21_0_RELEASE COMPUTERNAME=W64-IX-SLAVE88 COMSPEC=C:\windows\system32\cmd.exe CVS_RSH=ssh DXSDK_DIR=C:\Tools\sdks\dx10\ EDITOR=emacs.exe --no-window-system FP_NO_HOST_CHECK=NO FRAMEWORK35VERSION=v3.5 FRAMEWORKDIR=C:\Windows\Microsoft.NET\Framework64 FRAMEWORKVERSION=v2.0.50727 HG_SHARE_BASE_DIR=e:/builds/hg-shared HOME=c:/Users/cltbld HOMEDRIVE=C: HOMEPATH=\ HOSTTYPE=i686 INCLUDE=C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\ATLMFC\INCLUDE;C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\INCLUDE;C:\Tools\sdks\v7.0\include;;c:\tools\sdks\dx10\include INPUTRC=C:/mozilla-build/msys/etc/inputrc LIB=C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\ATLMFC\LIB\amd64;C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\LIB\amd64;C:\Tools\sdks\v7.0\lib\x64;;c:\tools\sdks\dx10\lib\x64 LIBPATH=C:\Windows\Microsoft.NET\Framework64\v3.5;C:\Windows\Microsoft.NET\Framework64\v2.0.50727;C:\Windows\Microsoft.NET\Framework64\v3.5;C:\Windows\Microsoft.NET\Framework64\v2.0.50727;C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\ATLMFC\LIB\amd64;C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\LIB\amd64; LOCALAPPDATA=C:\Users\cltbld\AppData\Local LOG="c:\tmp\buildbot-startup.log" LOGNAME=cltbld LOGONSERVER=\\W64-IX-SLAVE88 MACHTYPE=i686-pc-msys MAKE_MODE=unix MOZBUILDDIR=C:\mozilla-build\ MOZILLABUILD=C:\mozilla-build\ MOZILLA_REV=FIREFOX_21_0_RELEASE MOZ_CRASHREPORTER_NO_REPORT=1 MOZ_MAXWINSDK=999999 MOZ_MSVCVERSION=9 MOZ_OBJDIR=obj-firefox MOZ_PKG_PRETTYNAMES=1 MOZ_SIGN_CMD=python e:/builds/moz2_slave/rel-m-rel-w32_bld-000000000000/tools/release/signing/signtool.py --cachedir e:/builds/moz2_slave/rel-m-rel-w32_bld-000000000000/signing_cache -t e:/builds/moz2_slave/rel-m-rel-w32_bld-000000000000/token -n e:/builds/moz2_slave/rel-m-rel-w32_bld-000000000000/nonce -c e:/builds/moz2_slave/rel-m-rel-w32_bld-000000000000/tools/release/signing/host.cert -H signing3.srv.releng.scl3.mozilla.com:9120 MOZ_TOOLS=C:\mozilla-build\\moztools-x64 MOZ_UPDATE_CHANNEL=release MSVC10EXPRESSKEY=HKLM\SOFTWARE\Wow6432Node\Microsoft\VCExpress\10.0\Setup\VC MSVC10KEY=HKLM\SOFTWARE\Wow6432Node\Microsoft\VisualStudio\10.0\Setup\VC MSVC6KEY=HKLM\SOFTWARE\Wow6432Node\Microsoft\VisualStudio\6.0\Setup\Microsoft Visual C++ MSVC71KEY=HKLM\SOFTWARE\Wow6432Node\Microsoft\VisualStudio\7.1\Setup\VC MSVC8EXPRESSKEY=HKLM\SOFTWARE\Wow6432Node\Microsoft\VCExpress\8.0\Setup\VC MSVC8KEY=HKLM\SOFTWARE\Wow6432Node\Microsoft\VisualStudio\8.0\Setup\VC MSVC9EXPRESSKEY=HKLM\SOFTWARE\Wow6432Node\Microsoft\VCExpress\9.0\Setup\VC MSVC9KEY=HKLM\SOFTWARE\Wow6432Node\Microsoft\VisualStudio\9.0\Setup\VC MSVCEXPROOTKEY=HKLM\SOFTWARE\Wow6432Node\Microsoft\VCExpress MSVCROOTKEY=HKLM\SOFTWARE\Wow6432Node\Microsoft\VisualStudio MSYSTEM=MINGW32 NUMBER_OF_PROCESSORS=4 OLDPWD=c:/Users/cltbld OS=Windows_NT OSTYPE=msys PATH=C:\mozilla-build\nsis-2.46u;C:\mozilla-build\python27;C:\mozilla-build\buildbotve\scripts;C:\mozilla-build\msys\local\bin;c:\mozilla-build\wget;c:\mozilla-build\7zip;c:\mozilla-build\blat261\full;c:\mozilla-build\python;c:\mozilla-build\svn-win32-1.6.3\bin;c:\mozilla-build\upx203w;c:\mozilla-build\emacs-22.3\bin;c:\mozilla-build\info-zip;c:\mozilla-build\nsis-2.22;c:\mozilla-build\nsis-2.33u;c:\mozilla-build\nsis-2.46u;c:\mozilla-build\wix-351728;c:\mozilla-build\hg;c:\mozilla-build\python\Scripts;c:\mozilla-build\kdiff3;c:\mozilla-build\yasm;.;C:\mozilla-build\msys\local\bin;C:\mozilla-build\msys\mingw\bin;C:\mozilla-build\msys\bin;c:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN\amd64;c:\Windows\Microsoft.NET\Framework64\v3.5;c:\Windows\Microsoft.NET\Framework64\v3.5\Microsoft .NET Framework 3.5 (Pre-Release Version);c:\Windows\Microsoft.NET\Framework64\v2.0.50727;c:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\VCPackages;c:\Tools\msvs9\Common7\IDE;c:\Tools\msvs9\Common7\Tools;c:\Tools\msvs9\Common7\Tools\bin;c:\Tools\sdks\v7.0\bin\x64;c:\Tools\sdks\v7.0\bin\win64\x64;c:\Tools\sdks\v7.0\bin;c:\windows\System32;c:\windows;c:\windows\System32\Wbem;c:\mozilla-build\moztools-x64\bin;c:\mozilla-build\vim\vim72 PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC PDBSTR_PATH=/c/Program Files/Debugging Tools for Windows (x64)/srcsrv/pdbstr.exe POST_SYMBOL_UPLOAD_CMD=/usr/local/bin/post-symbol-upload.py PROCESSOR_ARCHITECTURE=x86 PROCESSOR_ARCHITEW6432=AMD64 PROCESSOR_IDENTIFIER=Intel64 Family 6 Model 30 Stepping 5, GenuineIntel PROCESSOR_LEVEL=6 PROCESSOR_REVISION=1e05 PROGRAMDATA=C:\ProgramData PROGRAMFILES=C:\Program Files (x86) PROGRAMFILES(X86)=C:\Program Files (x86) PROGRAMW6432=C:\Program Files PROMPT=$P$G PS1=\[\033]0;$MSYSTEM:\w\007 \033[32m\]\u@\h \[\033[33m\w\033[0m\] $ PSMODULEPATH=C:\windows\system32\WindowsPowerShell\v1.0\Modules\ PUBLIC=C:\Users\Public PWD=c:/Users/cltbld SDK2003SP1KEY=HKLM\SOFTWARE\Microsoft\MicrosoftSDK\InstalledSDKs\8F9E5EF3-A9A5-491B-A889-C58EFFECE8B3 SDK2003SP2KEY=HKLM\SOFTWARE\Microsoft\MicrosoftSDK\InstalledSDKs\D2FF9F89-8AA2-4373-8A31-C838BF4DBBE1 SDK61KEY=HKLM\SOFTWARE\Microsoft\Microsoft SDKs\Windows\v6.1 SDK6AKEY=HKLM\SOFTWARE\Microsoft\Microsoft SDKs\Windows\v6.0A SDK6KEY=HKLM\SOFTWARE\Microsoft\Microsoft SDKs\Windows\v6.0 SDK7KEY=HKLM\SOFTWARE\Microsoft\Microsoft SDKs\Windows\v7.0 SDKDIR=C:\Tools\sdks\v7.0\ SDKMINORVER=0 SDKROOTKEY=HKLM\SOFTWARE\Microsoft\MicrosoftSDK\InstalledSDKs SDKVER=7 SESSIONNAME=Console SHELL=C:/mozilla-build/msys/bin/sh SHLVL=1 SSH_AGENT_PID=2716 SSH_AUTH_SOCK=C:/Users/cltbld/AppData/Local/Temp/ssh-uXSBGh2660/agent.2660 SYMBOL_SERVER_HOST=symbolpush.mozilla.org SYMBOL_SERVER_PATH=/mnt/netapp/breakpad/symbols_ffx/ SYMBOL_SERVER_SSH_KEY=/c/Users/cltbld/.ssh/ffxbld_dsa SYMBOL_SERVER_USER=ffxbld SYSTEMDRIVE=C: SYSTEMROOT=C:\windows TEMP=C:/Users/cltbld/AppData/Local/Temp TEMPVC9DIR=C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\ TERM=cygwin TINDERBOX_OUTPUT=1 TMP=C:/Users/cltbld/AppData/Local/Temp USERDOMAIN=W64-IX-SLAVE88 USERNAME=cltbld USERPROFILE=C:\Users\cltbld VC10DIR=C:\Tools\msvs10\VC\ VC9DIR=C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\ VCINSTALLDIR=C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC VS100COMNTOOLS=C:\Tools\msvs10\Common7\Tools\ VS90COMNTOOLS=C:\Tools\msvs9\Common7\Tools\ VSINSTALLDIR=C:\Tools\msvs9 WIN64=1 WINCURVERKEY=HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion WINDIR=C:\windows WINDOWSSDKDIR=C:\Tools\sdks\v7.0\ WIX_351728_PATH=c:/mozilla-build/wix-351728 _=c:/mozilla-build/python/python using PTY: False retry: Calling <function run_with_timeout at 0x02BF6270> with args: (['python.exe', 'e:/builds/moz2_slave/rel-m-rel-w32_bld-000000000000/build/build/pymake/make.py', 'uploadsymbols'], 2460, None, None, False, True), kwargs: {}, attempt #1 Executing: ['python.exe', 'e:/builds/moz2_slave/rel-m-rel-w32_bld-000000000000/build/build/pymake/make.py', 'uploadsymbols'] make.py[0]: Entering directory 'e:\builds\moz2_slave\rel-m-rel-w32_bld-000000000000\build\obj-firefox' e:\builds\moz2_slave\rel-m-rel-w32_bld-000000000000\build\obj-firefox\Makefile:179:0$ /bin/sh e:/builds/moz2_slave/rel-m-rel-w32_bld-000000000000/build/toolkit/crashreporter/tools/upload_symbols.sh firefox-21.0-WINNT-20130507015204-x86-symbols.txt "./dist/win32/en-US/firefox-21.0.crashreporter-symbols-full.zip" Transferring symbols... ./dist/win32/en-US/firefox-21.0.crashreporter-symbols-full.zip debug1: Reading configuration data /c/Users/cltbld/.ssh/config debug1: Reading configuration data /etc/ssh/ssh_config debug1: Connecting to symbolpush.mozilla.org [63.245.217.193] port 22. debug1: Connection established. debug1: identity file /c/Users/cltbld/.ssh/ffxbld_dsa type -1 debug1: identity file /home/cltbld/.ssh/id_dsa type -1 debug1: Remote protocol version 2.0, remote software version OpenSSH_5.3 debug1: match: OpenSSH_5.3 pat OpenSSH* debug1: Enabling compatibility mode for protocol 2.0 debug1: Local version string SSH-2.0-OpenSSH_4.6 debug1: SSH2_MSG_KEXINIT sent debug1: SSH2_MSG_KEXINIT received debug1: kex: server->client aes128-cbc hmac-md5 none debug1: kex: client->server aes128-cbc hmac-md5 none debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP debug1: SSH2_MSG_KEX_DH_GEX_INIT sent debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY debug1: Host 'symbolpush.mozilla.org' is known and matches the RSA host key. debug1: Found key in /c/Users/cltbld/.ssh/known_hosts:19 debug1: ssh_rsa_verify: signature correct debug1: SSH2_MSG_NEWKEYS sent debug1: expecting SSH2_MSG_NEWKEYS debug1: SSH2_MSG_NEWKEYS received debug1: SSH2_MSG_SERVICE_REQUEST sent debug1: SSH2_MSG_SERVICE_ACCEPT received debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic debug1: Next authentication method: publickey debug1: Offering public key: /c/Users/cltbld/.ssh/id_rsa debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic debug1: Offering public key: /c/Users/cltbld/.ssh/id_dsa debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic debug1: Trying private key: /c/Users/cltbld/.ssh/ffxbld_dsa debug1: read PEM private key done: type DSA e:\builds\moz2_slave\rel-m-rel-w32_bld-000000000000\build\obj-firefox\Makefile:179:0: command '/bin/sh e:/builds/moz2_slave/rel-m-rel-w32_bld-000000000000/build/toolkit/crashreporter/tools/upload_symbols.sh firefox-21.0-WINNT-20130507015204-x86-symbols.txt "./dist/win32/en-US/firefox-21.0.crashreporter-symbols-full.zip"' failed, return code 1 debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic debug1: Trying private key: /home/cltbld/.ssh/id_dsa debug1: No more authentication methods to try. Permission denied (publickey,gssapi-keyex,gssapi-with-mic). lost connection retry: Failed, sleeping 1 seconds before retrying retry: Calling <function run_with_timeout at 0x02BF6270> with args: (['python.exe', 'e:/builds/moz2_slave/rel-m-rel-w32_bld-000000000000/build/build/pymake/make.py', 'uploadsymbols'], 2460, None, None, False, True), kwargs: {}, attempt #2 Executing: ['python.exe', 'e:/builds/moz2_slave/rel-m-rel-w32_bld-000000000000/build/build/pymake/make.py', 'uploadsymbols'] make.py[0]: Entering directory 'e:\builds\moz2_slave\rel-m-rel-w32_bld-000000000000\build\obj-firefox' e:\builds\moz2_slave\rel-m-rel-w32_bld-000000000000\build\obj-firefox\Makefile:179:0$ /bin/sh e:/builds/moz2_slave/rel-m-rel-w32_bld-000000000000/build/toolkit/crashreporter/tools/upload_symbols.sh firefox-21.0-WINNT-20130507015204-x86-symbols.txt "./dist/win32/en-US/firefox-21.0.crashreporter-symbols-full.zip" Transferring symbols... ./dist/win32/en-US/firefox-21.0.crashreporter-symbols-full.zip debug1: Reading configuration data /c/Users/cltbld/.ssh/config debug1: Reading configuration data /etc/ssh/ssh_config debug1: Connecting to symbolpush.mozilla.org [63.245.217.193] port 22. debug1: Connection established. debug1: identity file /c/Users/cltbld/.ssh/ffxbld_dsa type -1 debug1: identity file /home/cltbld/.ssh/id_dsa type -1 debug1: Remote protocol version 2.0, remote software version OpenSSH_5.3 debug1: match: OpenSSH_5.3 pat OpenSSH* debug1: Enabling compatibility mode for protocol 2.0 debug1: Local version string SSH-2.0-OpenSSH_4.6 debug1: SSH2_MSG_KEXINIT sent debug1: SSH2_MSG_KEXINIT received debug1: kex: server->client aes128-cbc hmac-md5 none debug1: kex: client->server aes128-cbc hmac-md5 none debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP debug1: SSH2_MSG_KEX_DH_GEX_INIT sent debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY debug1: Host 'symbolpush.mozilla.org' is known and matches the RSA host key. debug1: Found key in /c/Users/cltbld/.ssh/known_hosts:19 debug1: ssh_rsa_verify: signature correct debug1: SSH2_MSG_NEWKEYS sent debug1: expecting SSH2_MSG_NEWKEYS debug1: SSH2_MSG_NEWKEYS received debug1: SSH2_MSG_SERVICE_REQUEST sent debug1: SSH2_MSG_SERVICE_ACCEPT received debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic debug1: Next authentication method: publickey debug1: Offering public key: /c/Users/cltbld/.ssh/id_rsa debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic debug1: Offering public key: /c/Users/cltbld/.ssh/id_dsa debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic debug1: Trying private key: /c/Users/cltbld/.ssh/ffxbld_dsa debug1: read PEM private key done: type DSA debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic debug1: Trying private key: /home/cltbld/.ssh/id_dsa debug1: No more authentication methods to try. Permission denied (publickey,gssapi-keyex,gssapi-with-mic). lost connection e:\builds\moz2_slave\rel-m-rel-w32_bld-000000000000\build\obj-firefox\Makefile:179:0: command '/bin/sh e:/builds/moz2_slave/rel-m-rel-w32_bld-000000000000/build/toolkit/crashreporter/tools/upload_symbols.sh firefox-21.0-WINNT-20130507015204-x86-symbols.txt "./dist/win32/en-US/firefox-21.0.crashreporter-symbols-full.zip"' failed, return code 1 retry: Failed, sleeping 2 seconds before retrying retry: Calling <function run_with_timeout at 0x02BF6270> with args: (['python.exe', 'e:/builds/moz2_slave/rel-m-rel-w32_bld-000000000000/build/build/pymake/make.py', 'uploadsymbols'], 2460, None, None, False, True), kwargs: {}, attempt #3 Executing: ['python.exe', 'e:/builds/moz2_slave/rel-m-rel-w32_bld-000000000000/build/build/pymake/make.py', 'uploadsymbols'] make.py[0]: Entering directory 'e:\builds\moz2_slave\rel-m-rel-w32_bld-000000000000\build\obj-firefox' e:\builds\moz2_slave\rel-m-rel-w32_bld-000000000000\build\obj-firefox\Makefile:179:0$ /bin/sh e:/builds/moz2_slave/rel-m-rel-w32_bld-000000000000/build/toolkit/crashreporter/tools/upload_symbols.sh firefox-21.0-WINNT-20130507015204-x86-symbols.txt "./dist/win32/en-US/firefox-21.0.crashreporter-symbols-full.zip" Transferring symbols... ./dist/win32/en-US/firefox-21.0.crashreporter-symbols-full.zip debug1: Reading configuration data /c/Users/cltbld/.ssh/config debug1: Reading configuration data /etc/ssh/ssh_config debug1: Connecting to symbolpush.mozilla.org [63.245.217.193] port 22. debug1: Connection established. debug1: identity file /c/Users/cltbld/.ssh/ffxbld_dsa type -1 debug1: identity file /home/cltbld/.ssh/id_dsa type -1 debug1: Remote protocol version 2.0, remote software version OpenSSH_5.3 debug1: match: OpenSSH_5.3 pat OpenSSH* debug1: Enabling compatibility mode for protocol 2.0 debug1: Local version string SSH-2.0-OpenSSH_4.6 debug1: SSH2_MSG_KEXINIT sent debug1: SSH2_MSG_KEXINIT received debug1: kex: server->client aes128-cbc hmac-md5 none debug1: kex: client->server aes128-cbc hmac-md5 none debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP debug1: SSH2_MSG_KEX_DH_GEX_INIT sent debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY debug1: Host 'symbolpush.mozilla.org' is known and matches the RSA host key. debug1: Found key in /c/Users/cltbld/.ssh/known_hosts:19 debug1: ssh_rsa_verify: signature correct debug1: SSH2_MSG_NEWKEYS sent debug1: expecting SSH2_MSG_NEWKEYS debug1: SSH2_MSG_NEWKEYS received debug1: SSH2_MSG_SERVICE_REQUEST sent debug1: SSH2_MSG_SERVICE_ACCEPT received debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic debug1: Next authentication method: publickey debug1: Offering public key: /c/Users/cltbld/.ssh/id_rsa debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic debug1: Offering public key: /c/Users/cltbld/.ssh/id_dsa debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic debug1: Trying private key: /c/Users/cltbld/.ssh/ffxbld_dsa debug1: read PEM private key done: type DSA e:\builds\moz2_slave\rel-m-rel-w32_bld-000000000000\build\obj-firefox\Makefile:179:0: command '/bin/sh e:/builds/moz2_slave/rel-m-rel-w32_bld-000000000000/build/toolkit/crashreporter/tools/upload_symbols.sh firefox-21.0-WINNT-20130507015204-x86-symbols.txt "./dist/win32/en-US/firefox-21.0.crashreporter-symbols-full.zip"' failed, return code 1 debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic debug1: Trying private key: /home/cltbld/.ssh/id_dsa debug1: No more authentication methods to try. Permission denied (publickey,gssapi-keyex,gssapi-with-mic). lost connection retry: Failed, sleeping 4 seconds before retrying retry: Calling <function run_with_timeout at 0x02BF6270> with args: (['python.exe', 'e:/builds/moz2_slave/rel-m-rel-w32_bld-000000000000/build/build/pymake/make.py', 'uploadsymbols'], 2460, None, None, False, True), kwargs: {}, attempt #4 Executing: ['python.exe', 'e:/builds/moz2_slave/rel-m-rel-w32_bld-000000000000/build/build/pymake/make.py', 'uploadsymbols'] make.py[0]: Entering directory 'e:\builds\moz2_slave\rel-m-rel-w32_bld-000000000000\build\obj-firefox' e:\builds\moz2_slave\rel-m-rel-w32_bld-000000000000\build\obj-firefox\Makefile:179:0$ /bin/sh e:/builds/moz2_slave/rel-m-rel-w32_bld-000000000000/build/toolkit/crashreporter/tools/upload_symbols.sh firefox-21.0-WINNT-20130507015204-x86-symbols.txt "./dist/win32/en-US/firefox-21.0.crashreporter-symbols-full.zip" Transferring symbols... ./dist/win32/en-US/firefox-21.0.crashreporter-symbols-full.zip debug1: Reading configuration data /c/Users/cltbld/.ssh/config debug1: Reading configuration data /etc/ssh/ssh_config debug1: Connecting to symbolpush.mozilla.org [63.245.217.193] port 22. debug1: Connection established. debug1: identity file /c/Users/cltbld/.ssh/ffxbld_dsa type -1 debug1: identity file /home/cltbld/.ssh/id_dsa type -1 debug1: Remote protocol version 2.0, remote software version OpenSSH_5.3 debug1: match: OpenSSH_5.3 pat OpenSSH* debug1: Enabling compatibility mode for protocol 2.0 debug1: Local version string SSH-2.0-OpenSSH_4.6 debug1: SSH2_MSG_KEXINIT sent debug1: SSH2_MSG_KEXINIT received debug1: kex: server->client aes128-cbc hmac-md5 none debug1: kex: client->server aes128-cbc hmac-md5 none debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP debug1: SSH2_MSG_KEX_DH_GEX_INIT sent debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY debug1: Host 'symbolpush.mozilla.org' is known and matches the RSA host key. debug1: Found key in /c/Users/cltbld/.ssh/known_hosts:19 debug1: ssh_rsa_verify: signature correct debug1: SSH2_MSG_NEWKEYS sent debug1: expecting SSH2_MSG_NEWKEYS debug1: SSH2_MSG_NEWKEYS received debug1: SSH2_MSG_SERVICE_REQUEST sent debug1: SSH2_MSG_SERVICE_ACCEPT received debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic debug1: Next authentication method: publickey debug1: Offering public key: /c/Users/cltbld/.ssh/id_rsa debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic debug1: Offering public key: /c/Users/cltbld/.ssh/id_dsa debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic debug1: Trying private key: /c/Users/cltbld/.ssh/ffxbld_dsa debug1: read PEM private key done: type DSA debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic debug1: Trying private key: /home/cltbld/.ssh/id_dsa debug1: No more authentication methods to try. Permission denied (publickey,gssapi-keyex,gssapi-with-mic). lost connection e:\builds\moz2_slave\rel-m-rel-w32_bld-000000000000\build\obj-firefox\Makefile:179:0: command '/bin/sh e:/builds/moz2_slave/rel-m-rel-w32_bld-000000000000/build/toolkit/crashreporter/tools/upload_symbols.sh firefox-21.0-WINNT-20130507015204-x86-symbols.txt "./dist/win32/en-US/firefox-21.0.crashreporter-symbols-full.zip"' failed, return code 1 retry: Failed, sleeping 8 seconds before retrying retry: Calling <function run_with_timeout at 0x02BF6270> with args: (['python.exe', 'e:/builds/moz2_slave/rel-m-rel-w32_bld-000000000000/build/build/pymake/make.py', 'uploadsymbols'], 2460, None, None, False, True), kwargs: {}, attempt #5 Executing: ['python.exe', 'e:/builds/moz2_slave/rel-m-rel-w32_bld-000000000000/build/build/pymake/make.py', 'uploadsymbols'] make.py[0]: Entering directory 'e:\builds\moz2_slave\rel-m-rel-w32_bld-000000000000\build\obj-firefox' e:\builds\moz2_slave\rel-m-rel-w32_bld-000000000000\build\obj-firefox\Makefile:179:0$ /bin/sh e:/builds/moz2_slave/rel-m-rel-w32_bld-000000000000/build/toolkit/crashreporter/tools/upload_symbols.sh firefox-21.0-WINNT-20130507015204-x86-symbols.txt "./dist/win32/en-US/firefox-21.0.crashreporter-symbols-full.zip" Transferring symbols... ./dist/win32/en-US/firefox-21.0.crashreporter-symbols-full.zip debug1: Reading configuration data /c/Users/cltbld/.ssh/config debug1: Reading configuration data /etc/ssh/ssh_config debug1: Connecting to symbolpush.mozilla.org [63.245.217.193] port 22. debug1: Connection established. debug1: identity file /c/Users/cltbld/.ssh/ffxbld_dsa type -1 debug1: identity file /home/cltbld/.ssh/id_dsa type -1 debug1: Remote protocol version 2.0, remote software version OpenSSH_5.3 debug1: match: OpenSSH_5.3 pat OpenSSH* debug1: Enabling compatibility mode for protocol 2.0 debug1: Local version string SSH-2.0-OpenSSH_4.6 debug1: SSH2_MSG_KEXINIT sent debug1: SSH2_MSG_KEXINIT received debug1: kex: server->client aes128-cbc hmac-md5 none debug1: kex: client->server aes128-cbc hmac-md5 none debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP debug1: SSH2_MSG_KEX_DH_GEX_INIT sent debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY debug1: Host 'symbolpush.mozilla.org' is known and matches the RSA host key. debug1: Found key in /c/Users/cltbld/.ssh/known_hosts:19 debug1: ssh_rsa_verify: signature correct debug1: SSH2_MSG_NEWKEYS sent debug1: expecting SSH2_MSG_NEWKEYS debug1: SSH2_MSG_NEWKEYS received debug1: SSH2_MSG_SERVICE_REQUEST sent debug1: SSH2_MSG_SERVICE_ACCEPT received debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic debug1: Next authentication method: publickey debug1: Offering public key: /c/Users/cltbld/.ssh/id_rsa debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic debug1: Offering public key: /c/Users/cltbld/.ssh/id_dsa debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic debug1: Trying private key: /c/Users/cltbld/.ssh/ffxbld_dsa debug1: read PEM private key done: type DSA debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic debug1: Trying private key: /home/cltbld/.ssh/id_dsa debug1: No more authentication methods to try. Permission denied (publickey,gssapi-keyex,gssapi-with-mic). lost connection e:\builds\moz2_slave\rel-m-rel-w32_bld-000000000000\build\obj-firefox\Makefile:179:0: command '/bin/sh e:/builds/moz2_slave/rel-m-rel-w32_bld-000000000000/build/toolkit/crashreporter/tools/upload_symbols.sh firefox-21.0-WINNT-20130507015204-x86-symbols.txt "./dist/win32/en-US/firefox-21.0.crashreporter-symbols-full.zip"' failed, return code 1 retry: Giving up on <function run_with_timeout at 0x02BF6270> Unable to successfully run ['python.exe', 'e:/builds/moz2_slave/rel-m-rel-w32_bld-000000000000/build/build/pymake/make.py', 'uploadsymbols'] after 5 attempts program finished with exit code 1 elapsedTime=24.860000

It's possible that this was related to an IT change today, not a problem with the slave side keys.

It works fine now: cltbld@W64-IX-SLAVE88 ~/.ssh $ ssh -i ffxbld_dsa ffxbld@symbolpush.mozilla.org hostname symbols1.dmz.phx1.mozilla.com

https://tbpl.mozilla.org/php/getParsedLog.php?id=48390487&tree=Mozilla-Beta Disabled

Fixed by being reimaged for bug 1068922

Re-imaged and returned to production.