Android XUL mozilla-central nightly on 2012-04-06 03:12:22 PDT for push da0d07b5ca1e https://tbpl.mozilla.org/php/getParsedLog.php?id=10692174&tree=Firefox That^ nightly build seems to have hit an SSH issue ("ssh: connect to host aus2-staging.mozilla.org port 22: Connection refused"), but it had much spammier / more cryptic issues afterwards, which I suspect might be a bug in our scripts that is only exposed on exceptions like this. The chunk after that SSH failure has 5 repeated failures like this: {{{ retry: Calling <function run_with_timeout at 0xb7cf510c> with args: (['python', '/builds/slave/m-cen-andrd-xul-ntly/tools/scripts/updates/balrog-client.py', '--build-properties', 'buildprops_balrog.json', '--api-root', 'https://aus4-admin-dev.allizom.org', '--verbose', '--credentials-file', 'BuildSlaves.py'], 1260, None, None, False, True), kwargs: {}, attempt #1 Executing: ['python', '/builds/slave/m-cen-andrd-xul-ntly/tools/scripts/updates/balrog-client.py', '--build-properties', 'buildprops_balrog.json', '--api-root', 'https://aus4-admin-dev.allizom.org', '--verbose', '--credentials-file', 'BuildSlaves.py'] Traceback (most recent call last): File "/builds/slave/m-cen-andrd-xul-ntly/tools/scripts/updates/balrog-client.py", line 11, in <module> from balrog.client.cli import NightlyRunner File "/builds/slave/m-cen-andrd-xul-ntly/tools/lib/python/balrog/client/cli.py", line 7, in <module> from balrog.client.api import API File "/builds/slave/m-cen-andrd-xul-ntly/tools/lib/python/balrog/client/api.py", line 4, in <module> import requests File "/builds/slave/m-cen-andrd-xul-ntly/tools/lib/python/vendor/requests-0.10.8/requests/__init__.py", line 25, in <module> from . import utils File "/builds/slave/m-cen-andrd-xul-ntly/tools/lib/python/vendor/requests-0.10.8/requests/utils.py", line 232 return b"".join(L) ^ SyntaxError: invalid syntax retry: Failed, sleeping 1 seconds before retrying }}} ...followed by... {{{ retry: Giving up on <function run_with_timeout at 0xb7cf510c> Unable to successfully run ['python', '/builds/slave/m-cen-andrd-xul-ntly/tools/scripts/updates/balrog-client.py', '--build-properties', 'buildprops_balrog.json', '--api-root', 'https://aus4-admin-dev.allizom.org', '--verbose', '--credentials-file', 'BuildSlaves.py'] after 5 attempts program finished with exit code 1 }}} The SSH failure looks intermittent/connectivity-related (and is probably the proximate cause of bustage), but this syntax error looks like a real bug that we should fix.

Here's the relevant chunk of this log, from the failed SSH command up to the balrog-client.py syntax error issues (at the bottom).

Rail, can you take a look? It sounds like we might be using the wrong Python here? Or we have a buggy python-requests library?

looks like it's being run with python2.5 which doesn't support b"".

This should fix the problem

Comment on attachment 615728 [details] [diff] [review] set PYTHON26 env for android{,-xul} Seeing similar errors on Windows too.

Waiting for 2.7 on win32...

Comment on attachment 616584 [details] [diff] [review] set PYTHON26 env for android{,-xul}, win32 Once I'm done with bug 616584, I can deploy this.

Comment on attachment 616584 [details] [diff] [review] set PYTHON26 env for android{,-xul}, win32 http://hg.mozilla.org/build/buildbot-configs/rev/5e7816c35792

Worked fine for win32 nightlies. Android balrog submissions use proper python version and don't generate the reported error. However, it still can't submit updates. The corresponding issue is tracked in bug 746937,

Comment on attachment 616584 [details] [diff] [review] set PYTHON26 env for android{,-xul}, win32 http://hg.mozilla.org/build/buildbot-configs/rev/b89597d25dfd It causes win32 signing falures: 'bash' '-c' 'd:/mozilla-build/python27/python.exe e:/builds/moz2_slave/rel-m-beta-w32-bld/tools/release/signing/signtool.py --cachedir /e/builds/moz2_slave/rel-m-beta-w32-bld/signing_cache -t /e/builds/moz2_slave/rel-m-beta-w32-bld/token -n /e/builds/moz2_slave/rel-m-beta-w32-bld/nonce -c e:/builds/moz2_slave/rel-m-beta-w32-bld/tools/release/signing/host.cert -H signing1.build.scl1.mozilla.com:9120 -H signing2.build.scl1.mozilla.com:9120 -f gpg -f mar ""build/obj-firefox/dist/update/win32/en-US/firefox-13.0b2-13.0b3.partial.mar""' .... IOError: [Errno 2] No such file or directory: '\\build/obj-firefox/dist/update/win32/en-US/firefox-13.0b2-13.0b3.partial.mar\\'

The patch removes extra quotes. Passed staging.

Comment on attachment 624606 [details] [diff] [review] Remove extra quotes http://hg.mozilla.org/build/buildbotcustom/rev/b5ca2e87801b

Comment on attachment 616584 [details] [diff] [review] set PYTHON26 env for android{,-xul}, win32 http://hg.mozilla.org/build/buildbot-configs/rev/d518706b4646

in production 2012-05-17 0750 PDT

Comment on attachment 616584 [details] [diff] [review] set PYTHON26 env for android{,-xul}, win32 The win32 portion of this patch was backed out in bug 758064 due to win32 release l10n repack bustage.

maybe msys is doing it? http://www.mingw.org/wiki/Posix_path_conversion

Closing this one and reopening bug 758064, which has more details related to the win32 part of the bug.