Phil Ringnalda (:philor) Reporter
	Description • 12 years ago

mozilla-beta got an update Saturday, but mozilla-aurora did not, and when I looked at the hg log for browser/app/blocklist.xml, mozilla-central hasn't gotten one since September 15th.

	Rail Aliiev [:rail]
	Comment 1 • 12 years ago

http://buildbot-master35.srv.releng.scl3.mozilla.com:8001/builders/Linux%20mozilla-aurora%20blocklist%20update/builds/0/steps/run_script/logs/stdio

bash scripts/scripts/blocklist/sync-hg-blocklist.sh -b releases/mozilla-aurora -p firefox -u ffxbld -k '~cltbld/.ssh/ffxbld_dsa' -a
 in dir /builds/slave/m-aurora-lnx-blocklistupdate/. (timeout 1200 secs)
 watching logfiles {}
 argv: ['bash', 'scripts/scripts/blocklist/sync-hg-blocklist.sh', '-b', 'releases/mozilla-aurora', '-p', 'firefox', '-u', 'ffxbld', '-k', '~cltbld/.ssh/ffxbld_dsa', '-a']
 environment:
  CCACHE_HASHDIR=
  G_BROKEN_FILENAMES=1
  HISTCONTROL=ignoredups
  HISTSIZE=1000
  HOME=/home/cltbld
  HOSTNAME=bld-linux64-ec2-038.build.aws-us-west-1.mozilla.com
  LESSOPEN=|/usr/bin/lesspipe.sh %s
  LOGNAME=cltbld
  MAIL=/var/spool/mail/cltbld
  PATH=/usr/local/bin:/usr/lib64/ccache:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/cltbld/bin
  PROPERTIES_FILE=/builds/slave/m-aurora-lnx-blocklistupdate/buildprops.json
  PWD=/builds/slave/m-aurora-lnx-blocklistupdate
  SHELL=/bin/bash
  SHLVL=1
  TERM=linux
  USER=cltbld
  _=/tools/buildbot/bin/python
 using PTY: False
--2012-10-06 03:02:25--  http://hg.mozilla.org/releases/mozilla-aurora/raw-file/default/browser/config/version.txt
Resolving hg.mozilla.org... 10.22.74.30
Connecting to hg.mozilla.org|10.22.74.30|:80... connected.
HTTP request sent, awaiting response... 200 Script output follows
Length: 7 [text/plain]
Saving to: `version.txt'

     0K                                                       100%  767K=0s

2012-10-06 03:02:26 (767 KB/s) - `version.txt' saved [7/7]

--2012-10-06 03:02:26--  https://addons.mozilla.org/blocklist/3/%7Bec8030f7-c20a-464f-9b0e-13a3a9e97384%7D/17.0a2/Firefox/20090105024647/blocklist-sync/en-US/nightly/blocklist-sync/default/default/
Resolving addons.mozilla.org... 63.245.217.112, 2620:101:8008:5::2:2
Connecting to addons.mozilla.org|63.245.217.112|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 30379 (30K) [text/xml]
Saving to: `blocklist_amo.xml'

     0K .......... .......... .........                       100% 1.31M=0.02s

2012-10-06 03:02:26 (1.31 MB/s) - `blocklist_amo.xml' saved [30379/30379]

--2012-10-06 03:02:26--  http://hg.mozilla.org/releases/mozilla-aurora/raw-file/default/browser/app/blocklist.xml
Resolving hg.mozilla.org... 10.22.74.30
Connecting to hg.mozilla.org|10.22.74.30|:80... connected.
HTTP request sent, awaiting response... 200 Script output follows
Length: 29111 (28K) [text/xml]
Saving to: `blocklist_hg.xml'

     0K .......... .......... ........                        100% 5.67M=0.005s

2012-10-06 03:02:26 (5.67 MB/s) - `blocklist_hg.xml' saved [29111/29111]

Blocklist files differ, updating hg.
hg clone http://hg.mozilla.org/releases/mozilla-aurora blocklist
requesting all changes
adding changesets
adding manifests
adding file changes
transaction abort!
rollback completed
abort: connection ended unexpectedly
ERROR hg clone exited with a non-zero exit code: 255
program finished with exit code 255
elapsedTime=475.939200

	Chris Cooper [:coop] (he/him) Assignee
	Comment 2 • 12 years ago

I think this is because the script is naively cloning from the main repo, but should be leveraging hgtool.py and the releng mirrors instead.

	Phil Ringnalda (:philor) Reporter
	Comment 3 • 12 years ago

It would be extra lovely if we could have that be "-t) USE_HGTOOL=true;;" since for bug 791492 I need to write docs for relman saying how to manually update, and "save a copy of sync-hg-blocklist.sh, chmod 755, ./sync-hg-blocklist.sh -p firefox -b releases/mozilla-release -u youruser -k yourkey etc." would be nicer than having it require a full clone of tools/ and especially nicer than requiring that a --shared-dir exist and be in a particular place.

	Chris Cooper [:coop] (he/him) Assignee
	Comment 4 • 12 years ago

(In reply to Phil Ringnalda (:philor) from comment #3)
> It would be extra lovely if we could have that be "-t) USE_HGTOOL=true;;"
> since for bug 791492 I need to write docs for relman saying how to manually
> update, and "save a copy of sync-hg-blocklist.sh, chmod 755,
> ./sync-hg-blocklist.sh -p firefox -b releases/mozilla-release -u youruser -k
> yourkey etc." would be nicer than having it require a full clone of tools/
> and especially nicer than requiring that a --shared-dir exist and be in a
> particular place.

Given that sync-hg-blocklist.sh is part of the tools repo, I'm not sure how onerous it would be to require the full checkout. 

relman certainly wouldn't need to use hgtool.py, although it might still make their life easier to do so given the bundle download speed. 

For the record, my plan is 4-fold:
* check for hgtool.py specified on the command-line
* check for hgtool.py in the PATH
* check for hgtool.py in the current tools/ checkout (assuming blocklist script is being run from within that checkout)
* default to regular clone

	Phil Ringnalda (:philor) Reporter
	Comment 5 • 12 years ago

Actually, I was looking at the wrong thing, and what I should have been looking at is the way REPODIR is hardcoded, since the way I picture them doing it is having a local clone from which they push a merge to mozilla-release, then an hour or two later when that seems to be building reasonably, pushing a blocklist update from that repo.

	Phil Ringnalda (:philor) Reporter
	Comment 6 • 12 years ago

Score for the week: mozilla-central and mozilla-aurora should have been updated, but weren't; mozilla-beta, mozilla-release and mozilla-esr10 were already up to date.

	Phil Ringnalda (:philor) Reporter
	Comment 7 • 12 years ago

Score for the week: mozilla-central and mozilla-aurora should have been updated, but weren't, so I did them manually-automatically; mozilla-beta, mozilla-release and mozilla-esr10 got updated automatically.

	Justin Wood (:Callek)
	Comment 8 • 12 years ago

Failed:
bld-linux64-ec2-057 	comm-central-linux-blocklistupdate 	1 	bm35-build1 	2 	2012-11-03 03:07:25 	2012-11-03 03:10:06 	0:02:41
bld-linux64-ec2-076 	mozilla-aurora-linux-blocklistupdate 	1 	bm35-build1 	2 	2012-11-03 03:07:25 	2012-11-03 03:15:58 	0:08:33
bld-linux64-ec2-020 	comm-aurora-linux-blocklistupdate 	1 	bm35-build1 	2 	2012-11-03 03:07:24 	2012-11-03 03:10:16 	0:02:52
bld-linux64-ec2-022 	mozilla-central-linux-blocklistupdate 	2 	bm35-build1 	2 	2012-11-03 03:07:24 	2012-11-03 03:15:46 	0:08:22

e.g. for m-c:

Blocklist files differ, updating hg.
hg clone http://hg.mozilla.org/mozilla-central blocklist
requesting all changes
adding changesets
adding manifests
adding file changes
added 112210 changesets with 632036 changes to 109148 files (+7 heads)
updating to branch default
68723 files updated, 0 files merged, 0 files removed, 0 files unresolved
hg -R blocklist pull
pulling from http://hg.mozilla.org/mozilla-central
searching for changes
no changes found
hg -R blocklist update -C default
0 files updated, 0 files merged, 0 files removed, 0 files unresolved
hg -R blocklist commit -u "ffxbld" -m "No bug, Automated blocklist update from host bld-linux64-ec2-022 - a=blocklist-update"
hg -R blocklist push -e "ssh -l ffxbld -i ~cltbld/.ssh/ffxbld_dsa" ssh://hg.mozilla.org/mozilla-central
pushing to ssh://hg.mozilla.org/mozilla-central
remote: ssh: connect to host hg.mozilla.org port 22: Connection timed out
abort: no suitable response from remote hg!
ERROR hg push exited with exit code: 255, probably raced another changeset
hg -R blocklist rollback
repository tip rolled back to revision 112209 (undo commit)
working directory now based on revision 112209
program finished with exit code 255

	Rail Aliiev [:rail]
	Comment 9 • 12 years ago

> remote: ssh: connect to host hg.mozilla.org port 22: Connection timed out

Sounds like a netflow issue. I'll file a bug for this.

	Chris Cooper [:coop] (he/him) Assignee
	Comment 10 • 12 years ago

Fixed up the usage string and added some consistency around how squiggly brackets are used, but the meat of this is adding support for hgtool.py, either specified on the command line or present in the PATH. The usual --mirror and --bundle params will work with hgtool.py.

I still need to update the builder to use the new options, but the expected syntax will be, e.g. for aurora:

bash scripts/scripts/blocklist/sync-hg-blocklist.sh -b releases/mozilla-aurora \
 -p firefox -u ffxbld -k '~cltbld/.ssh/ffxbld_dsa' -a \
 --mirror http://hg-internal.dmz.scl3.mozilla.com/releases/mozilla-aurora \
 --bundle http://ftp.mozilla.org/pub/mozilla.org/firefox/bundles/mozilla-aurora.hg \
 --hgtool scripts/buildfarm/utils/hgtool.py

philor: as a "frequent" user of this script, not sure if you want to have a look too.

	Justin Wood (:Callek)
	Comment 11 • 12 years ago

Comment on attachment 679363 [details] [diff] [review]
Use hgtool.py when present

Review of attachment 679363 [details] [diff] [review]:
-----------------------------------------------------------------

::: scripts/blocklist/sync-hg-blocklist.sh
@@ +210,3 @@
>      PUSH_STATUS=$?
> +    if [ ${PUSH_STATUS} != 0 ]; then
> +        echo "ERROR hg push exited with exit code: ${PUSH_STATUS}, probably raced another changeset"

While all this is a great and needed change, and should speed up our total blocklist update runtime, because of this error, and subsequent commands, this patch does not address the overall problem here (seen sometimes) of blocklist update actually racing another commit and just (silently) failing.

We should do something similar to what our tagging code does I think, and/or just pull/update a few times trying to re-do the change before we give up.

That said, that does not block this improvement landing! r+

	Phil Ringnalda (:philor) Reporter
	Comment 12 • 12 years ago

Comment on attachment 679363 [details] [diff] [review]
Use hgtool.py when present

I'm a little concerned that updating the usage string will cause someone to notice that I didn't update it when I added -r, but other than that looks fine to me :)

	Chris Cooper [:coop] (he/him) Assignee
	Comment 13 • 12 years ago

(In reply to Justin Wood (:Callek) from comment #11)
> While all this is a great and needed change, and should speed up our total
> blocklist update runtime, because of this error, and subsequent commands,
> this patch does not address the overall problem here (seen sometimes) of
> blocklist update actually racing another commit and just (silently) failing.

Agreed, but the use of hgtool.py with mirrors and bundles *should* cut down on the turnaround time between initial pull and final push.

	Chris Cooper [:coop] (he/him) Assignee
	Comment 14 • 12 years ago

Comment on attachment 679363 [details] [diff] [review]
Use hgtool.py when present

https://hg.mozilla.org/build/tools/rev/5e66cb6c7057

	Chris Cooper [:coop] (he/him) Assignee
	Comment 15 • 12 years ago

There's a successful staging build with these changes here:

http://dev-master01.build.scl1.mozilla.com:8044/builders/Linux%20mozilla-central%20blocklist%20update/builds/0

...although the actual push was disabled (intentionally).

	Chris Cooper [:coop] (he/him) Assignee
	Comment 16 • 12 years ago

Comment on attachment 680250 [details] [diff] [review]
Add args for hgtool, mirror, and bundle to blocklist builder

https://hg.mozilla.org/build/buildbotcustom/rev/de9e02f82370

	Phil Ringnalda (:philor) Reporter
	Comment 17 • 12 years ago

Score for the week: mozilla-beta and mozilla-esr17 were updated, mozilla-central and mozilla-aurora and mozilla-release and mozilla-esr10 were not but should have been.

	Justin Wood (:Callek)
	Comment 18 • 12 years ago

Ok, with the attached patches (after reconf) I ran c-a's blocklist update script run again and we failed:

(attaching log here so people outside of releng can look -- I'm too tired atm to investigate)

bash scripts/scripts/blocklist/sync-hg-blocklist.sh -b releases/comm-aurora --hgtool scripts/buildfarm/utils/hgtool.py --mirror http://hg-internal.dmz.scl3.mozilla.com/releases/comm-aurora --bundle http://ftp.mozilla.org/pub/mozilla.org/thunderbird/bundles/comm-aurora.hg -p thunderbird -u tbirdbld -k '~cltbld/.ssh/tbirdbld_dsa' -a
 in dir /builds/slave/c-aurora-lnx-blocklistupdate/. (timeout 1200 secs)
 watching logfiles {}
 argv: ['bash', 'scripts/scripts/blocklist/sync-hg-blocklist.sh', '-b', 'releases/comm-aurora', '--hgtool', 'scripts/buildfarm/utils/hgtool.py', '--mirror', 'http://hg-internal.dmz.scl3.mozilla.com/releases/comm-aurora', '--bundle', 'http://ftp.mozilla.org/pub/mozilla.org/thunderbird/bundles/comm-aurora.hg', '-p', 'thunderbird', '-u', 'tbirdbld', '-k', '~cltbld/.ssh/tbirdbld_dsa', '-a']
 environment:
  CCACHE_HASHDIR=
  G_BROKEN_FILENAMES=1
  HISTCONTROL=ignoredups
  HISTSIZE=1000
  HOME=/home/cltbld
  HOSTNAME=bld-linux64-ec2-054.build.aws-us-west-1.mozilla.com
  LESSOPEN=|/usr/bin/lesspipe.sh %s
  LOGNAME=cltbld
  MAIL=/var/spool/mail/cltbld
  PATH=/usr/local/bin:/usr/lib64/ccache:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/cltbld/bin
  PROPERTIES_FILE=/builds/slave/c-aurora-lnx-blocklistupdate/buildprops.json
  PWD=/builds/slave/c-aurora-lnx-blocklistupdate
  SHELL=/bin/bash
  SHLVL=1
  TERM=linux
  USER=cltbld
  _=/tools/buildbot/bin/python
 using PTY: False
--2012-11-10 23:10:14--  http://hg.mozilla.org/releases/comm-aurora/raw-file/default/mail/config/version.txt
Resolving hg.mozilla.org... 10.22.74.30
Connecting to hg.mozilla.org|10.22.74.30|:80... connected.
HTTP request sent, awaiting response... 200 Script output follows
Length: 7 [text/plain]
Saving to: `version.txt'

     0K                                                       100%  765K=0s

2012-11-10 23:10:14 (765 KB/s) - `version.txt' saved [7/7]

--2012-11-10 23:10:14--  https://addons.mozilla.org/blocklist/3/%7B3550f703-e582-4d05-9a08-453d09bdfdc6%7D/18.0a2/Thunderbird/20090105024647/blocklist-sync/en-US/nightly/blocklist-sync/default/default/
Resolving addons.mozilla.org... 63.245.217.112, 2620:101:8008:5::2:2
Connecting to addons.mozilla.org|63.245.217.112|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/xml]
Saving to: `blocklist_amo.xml'

     0K .......... .......... .....                            1022K=0.02s

2012-11-10 23:10:15 (1022 KB/s) - `blocklist_amo.xml' saved [25962]

--2012-11-10 23:10:15--  http://hg.mozilla.org/releases/comm-aurora/raw-file/default/mail/app/blocklist.xml
Resolving hg.mozilla.org... 10.22.74.30
Connecting to hg.mozilla.org|10.22.74.30|:80... connected.
HTTP request sent, awaiting response... 200 Script output follows
Length: 22368 (22K) [text/xml]
Saving to: `blocklist_hg.xml'

     0K .......... .......... .                               100% 3.82M=0.006s

2012-11-10 23:10:15 (3.82 MB/s) - `blocklist_hg.xml' saved [22368/22368]

Blocklist files differ, updating hg.
scripts/buildfarm/utils/hgtool.py --mirror http://hg-internal.dmz.scl3.mozilla.com/releases/comm-aurora --bundle http://ftp.mozilla.org/pub/mozilla.org/thunderbird/bundles/comm-aurora.hg http://hg.mozilla.org/releases/comm-aurora blocklist
Attempting to initialize clone with bundles
command: START
command: hg init /builds/slave/c-aurora-lnx-blocklistupdate/blocklist
command: cwd: /builds/slave/c-aurora-lnx-blocklistupdate
command: output:
command: END (0.37s elapsed)

Trying to use bundle http://ftp.mozilla.org/pub/mozilla.org/thunderbird/bundles/comm-aurora.hg
command: START
command: hg unbundle http://ftp.mozilla.org/pub/mozilla.org/thunderbird/bundles/comm-aurora.hg
command: cwd: /builds/slave/c-aurora-lnx-blocklistupdate/blocklist
command: output:
adding changesets
adding manifests
adding file changes
added 12552 changesets with 52577 changes to 11038 files (+6 heads)
(run 'hg heads' to see heads)
command: END (75.39s elapsed)

command: START
command: hg -q version
command: cwd: /builds/slave/c-aurora-lnx-blocklistupdate
command: output:
Mercurial Distributed SCM (version 2.1.1)

command: END (0.34 elapsed)

command: START
command: hg pull -b releases/comm-aurora http://hg-internal.dmz.scl3.mozilla.com/releases/comm-aurora
command: cwd: /builds/slave/c-aurora-lnx-blocklistupdate/blocklist
command: output:
pulling from http://hg-internal.dmz.scl3.mozilla.com/releases/comm-aurora
abort: unknown branch 'releases/comm-aurora'!
command: ERROR
Traceback (most recent call last):
  File "scripts/buildfarm/utils/../../lib/python/util/commands.py", line 40, in run_cmd
    return subprocess.check_call(cmd, **kwargs)
  File "/usr/lib64/python2.6/subprocess.py", line 502, in check_call
    raise CalledProcessError(retcode, cmd)
CalledProcessError: Command '['hg', 'pull', '-b', u'releases/comm-aurora', 'http://hg-internal.dmz.scl3.mozilla.com/releases/comm-aurora']' returned non-zero exit status 255
command: END (0.57s elapsed)

Problem pulling from mirror http://hg-internal.dmz.scl3.mozilla.com/releases/comm-aurora
Traceback (most recent call last):
  File "scripts/buildfarm/utils/../../lib/python/util/hg.py", line 224, in pull
    return pull(mirror, dest, update_dest=update_dest, **kwargs)
  File "scripts/buildfarm/utils/../../lib/python/util/hg.py", line 246, in pull
    run_cmd(cmd, cwd=dest)
  File "scripts/buildfarm/utils/../../lib/python/util/commands.py", line 40, in run_cmd
    return subprocess.check_call(cmd, **kwargs)
  File "/usr/lib64/python2.6/subprocess.py", line 502, in check_call
    raise CalledProcessError(retcode, cmd)
CalledProcessError: Command '['hg', 'pull', '-b', u'releases/comm-aurora', 'http://hg-internal.dmz.scl3.mozilla.com/releases/comm-aurora']' returned non-zero exit status 255
Pulling from mirrors failed; falling back to http://hg.mozilla.org/releases/comm-aurora
command: START
command: hg -q version
command: cwd: /builds/slave/c-aurora-lnx-blocklistupdate
command: output:
Mercurial Distributed SCM (version 2.1.1)

command: END (0.32 elapsed)

command: START
command: hg pull -b releases/comm-aurora http://hg.mozilla.org/releases/comm-aurora
command: cwd: /builds/slave/c-aurora-lnx-blocklistupdate/blocklist
command: output:
pulling from http://hg.mozilla.org/releases/comm-aurora
abort: unknown branch 'releases/comm-aurora'!
command: ERROR
Traceback (most recent call last):
  File "scripts/buildfarm/utils/../../lib/python/util/commands.py", line 40, in run_cmd
    return subprocess.check_call(cmd, **kwargs)
  File "/usr/lib64/python2.6/subprocess.py", line 502, in check_call
    raise CalledProcessError(retcode, cmd)
CalledProcessError: Command '['hg', 'pull', '-b', u'releases/comm-aurora', 'http://hg.mozilla.org/releases/comm-aurora']' returned non-zero exit status 255
command: END (0.60s elapsed)

Problem unbundling/pulling from http://ftp.mozilla.org/pub/mozilla.org/thunderbird/bundles/comm-aurora.hg
Traceback (most recent call last):
  File "scripts/buildfarm/utils/../../lib/python/util/hg.py", line 146, in clone
    mirrors=mirrors, revision=revision, branch=branch)
  File "scripts/buildfarm/utils/../../lib/python/util/hg.py", line 246, in pull
    run_cmd(cmd, cwd=dest)
  File "scripts/buildfarm/utils/../../lib/python/util/commands.py", line 40, in run_cmd
    return subprocess.check_call(cmd, **kwargs)
  File "/usr/lib64/python2.6/subprocess.py", line 502, in check_call
    raise CalledProcessError(retcode, cmd)
CalledProcessError: Command '['hg', 'pull', '-b', u'releases/comm-aurora', 'http://hg.mozilla.org/releases/comm-aurora']' returned non-zero exit status 255
Using bundles failed; falling back to clone
Attempting to clone from mirrors
Cloning from http://hg-internal.dmz.scl3.mozilla.com/releases/comm-aurora
command: START
command: hg clone http://hg-internal.dmz.scl3.mozilla.com/releases/comm-aurora /builds/slave/c-aurora-lnx-blocklistupdate/blocklist
command: cwd: /builds/slave/c-aurora-lnx-blocklistupdate
command: output:
requesting all changes
adding changesets
adding manifests
adding file changes
added 12552 changesets with 52577 changes to 11038 files (+6 heads)
updating to branch default
7969 files updated, 0 files merged, 0 files removed, 0 files unresolved
command: END (48.10s elapsed)

command: START
command: hg branch
command: cwd: /builds/slave/c-aurora-lnx-blocklistupdate/blocklist
command: output:
default

command: END (0.39 elapsed)

command: START
command: hg update -C releases/comm-aurora
command: cwd: /builds/slave/c-aurora-lnx-blocklistupdate/blocklist
command: output:
abort: unknown revision 'releases/comm'!
command: ERROR
Traceback (most recent call last):
  File "scripts/buildfarm/utils/../../lib/python/util/commands.py", line 40, in run_cmd
    return subprocess.check_call(cmd, **kwargs)
  File "/usr/lib64/python2.6/subprocess.py", line 502, in check_call
    raise CalledProcessError(retcode, cmd)
CalledProcessError: Command '['hg', 'update', '-C', u'releases/comm-aurora']' returned non-zero exit status 255
command: END (0.46s elapsed)

Problem cloning from mirror http://hg-internal.dmz.scl3.mozilla.com/releases/comm-aurora
Traceback (most recent call last):
  File "scripts/buildfarm/utils/../../lib/python/util/hg.py", line 160, in clone
    update_dest=update_dest, clone_by_rev=clone_by_rev)
  File "scripts/buildfarm/utils/../../lib/python/util/hg.py", line 192, in clone
    return update(dest, branch, revision)
  File "scripts/buildfarm/utils/../../lib/python/util/hg.py", line 107, in update
    run_cmd(cmd, cwd=dest)
  File "scripts/buildfarm/utils/../../lib/python/util/commands.py", line 40, in run_cmd
    return subprocess.check_call(cmd, **kwargs)
  File "/usr/lib64/python2.6/subprocess.py", line 502, in check_call
    raise CalledProcessError(retcode, cmd)
CalledProcessError: Command '['hg', 'update', '-C', u'releases/comm-aurora']' returned non-zero exit status 255
Pulling from mirrors failed; falling back to http://hg.mozilla.org/releases/comm-aurora
command: START
command: hg path default
command: cwd: /builds/slave/c-aurora-lnx-blocklistupdate/blocklist
command: output:
http://hg.mozilla.org/releases/comm-aurora

command: END (0.38 elapsed)

command: START
command: hg -q version
command: cwd: /builds/slave/c-aurora-lnx-blocklistupdate
command: output:
Mercurial Distributed SCM (version 2.1.1)

command: END (0.33 elapsed)

command: START
command: hg pull -b releases/comm-aurora http://hg.mozilla.org/releases/comm-aurora
command: cwd: /builds/slave/c-aurora-lnx-blocklistupdate/blocklist
command: output:
pulling from http://hg.mozilla.org/releases/comm-aurora
abort: unknown branch 'releases/comm-aurora'!
command: ERROR
Traceback (most recent call last):
  File "scripts/buildfarm/utils/../../lib/python/util/commands.py", line 40, in run_cmd
    return subprocess.check_call(cmd, **kwargs)
  File "/usr/lib64/python2.6/subprocess.py", line 502, in check_call
    raise CalledProcessError(retcode, cmd)
CalledProcessError: Command '['hg', 'pull', '-b', u'releases/comm-aurora', 'http://hg.mozilla.org/releases/comm-aurora']' returned non-zero exit status 255
command: END (0.58s elapsed)

Error pulling changes into /builds/slave/c-aurora-lnx-blocklistupdate/blocklist from http://hg.mozilla.org/releases/comm-aurora; clobbering
command: START
command: hg clone http://hg.mozilla.org/releases/comm-aurora /builds/slave/c-aurora-lnx-blocklistupdate/blocklist
command: cwd: /builds/slave/c-aurora-lnx-blocklistupdate
command: output:
requesting all changes
adding changesets
adding manifests
adding file changes
added 12552 changesets with 52577 changes to 11038 files (+6 heads)
updating to branch default
7969 files updated, 0 files merged, 0 files removed, 0 files unresolved
command: END (41.91s elapsed)

command: START
command: hg branch
command: cwd: /builds/slave/c-aurora-lnx-blocklistupdate/blocklist
command: output:
default

command: END (0.37 elapsed)

command: START
command: hg update -C releases/comm-aurora
command: cwd: /builds/slave/c-aurora-lnx-blocklistupdate/blocklist
command: output:
abort: unknown revision 'releases/comm'!
command: ERROR
Traceback (most recent call last):
  File "scripts/buildfarm/utils/../../lib/python/util/commands.py", line 40, in run_cmd
    return subprocess.check_call(cmd, **kwargs)
  File "/usr/lib64/python2.6/subprocess.py", line 502, in check_call
    raise CalledProcessError(retcode, cmd)
CalledProcessError: Command '['hg', 'update', '-C', u'releases/comm-aurora']' returned non-zero exit status 255
command: END (0.45s elapsed)

Traceback (most recent call last):
  File "scripts/buildfarm/utils/hgtool.py", line 89, in <module>
    bundles=options.bundles)
  File "scripts/buildfarm/utils/../../lib/python/util/hg.py", line 434, in mercurial
    bundles=bundles, clone_by_rev=clone_by_rev)
  File "scripts/buildfarm/utils/../../lib/python/util/hg.py", line 173, in clone
    update_dest=update_dest, clone_by_rev=clone_by_rev)
  File "scripts/buildfarm/utils/../../lib/python/util/hg.py", line 434, in mercurial
    bundles=bundles, clone_by_rev=clone_by_rev)
  File "scripts/buildfarm/utils/../../lib/python/util/hg.py", line 192, in clone
    return update(dest, branch, revision)
  File "scripts/buildfarm/utils/../../lib/python/util/hg.py", line 107, in update
    run_cmd(cmd, cwd=dest)
  File "scripts/buildfarm/utils/../../lib/python/util/commands.py", line 40, in run_cmd
    return subprocess.check_call(cmd, **kwargs)
  File "/usr/lib64/python2.6/subprocess.py", line 502, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['hg', 'update', '-C', u'releases/comm-aurora']' returned non-zero exit status 255
ERROR hg clone exited with a non-zero exit code: 1

	Justin Wood (:Callek)
	Comment 19 • 12 years ago

(In reply to Justin Wood (:Callek) from comment #18)
> Ok, with the attached patches (after reconf) I ran c-a's blocklist update
> script run again and we failed:
> 

comm-central failed similarly (http://buildbot-master35.srv.releng.scl3.mozilla.com:8001/builders/TB%20Linux%20comm-central%20blocklist%20update/builds/3/steps/run_script/logs/stdio)

	Phil Ringnalda (:philor) Reporter
	Comment 20 • 12 years ago

Score for the week: 0.

	Chris Cooper [:coop] (he/him) Assignee
	Comment 21 • 12 years ago

hgtool.py will pull the branch out of buildprops.json if present, but won't override a branch set on the command-line.

	Justin Wood (:Callek)
	Comment 22 • 12 years ago

Comment on attachment 683098 [details] [diff] [review]
Pass branch to hgtool.py

Review of attachment 683098 [details] [diff] [review]:
-----------------------------------------------------------------

::: scripts/blocklist/sync-hg-blocklist.sh
@@ +155,5 @@
>      if [ ! -d ${REPODIR} ]; then
>          CLONE_CMD=""
>          if [ -f "${HGTOOL}" ]; then
> +	    # Need to pass the default branch here to avoid pollution from buildprops.json
> +	    # when hgtool.py is run in production. 

nit: whitespace at EOL

	Chris Cooper [:coop] (he/him) Assignee
	Comment 23 • 12 years ago

Comment on attachment 683098 [details] [diff] [review]
Pass branch to hgtool.py

https://hg.mozilla.org/build/tools/rev/4307d455e060

	Chris Cooper [:coop] (he/him) Assignee
	Comment 24 • 12 years ago

I think we're good here (finally).