We have stage.m.o for both ssh uploads and http(s) downloads, but after moving to S3 we'll have different hosts for each job.

There's several things going on here * the new staging server for upload requires a --bucket-prefix to talk to the right S3 bucket, so we need to pass this around a lot of places (thought this was going to be less, otherwise I'd have asked oremj to fix that) * those are defined in attachment 8651057 [details] [diff] [review], then used in several places * ReleaseBuildFactory gets an ftpServer arg, so that it can download previous releases for partial mar generation (not for long, long live funsize!) * make sure we don't request paths with // in them. Apache ignored that, the new system doesn't (eg oldCandidatesDir) * stop uploading info.txt files in buildN dir for fennec, TODO - adjust the Socorro ftp scraper to cope * stop using hardcoded post_upload commands in the source factory etc * ReleaseUpdatesFactory no longer needs a stagingServer arg, it points at ftpServer for all downloads * pass a new --ftp-server arg to l10n repacks, so they can get the en-US build and old complete mars In the release config we'll have this for staging: releaseConfig['ftpServer'] = 'ftp.stage.mozaws.net' releaseConfig['stagingServer'] = 'upload.ffxbld.productdelivery.stage.mozaws.net'

* Patcher land: * lots of renaming of stagingServer to ftpServer * more removals of // in request paths * use candidateDir rather than generate it again in many places * dump the ancient and crappy tests * Balrog land: * renaming * we're going to use <app>/candidates/ and drop the symlinks in <app>/nightly that point back there, so drop the mobile hack in _getFileUrls() * L10n land ('tis a mythical place full of dragons): * avoid // some more when getting mar and mbsdiff * add a DOWNLOAD_HOST variable for lightning, since that has yet another way of setting the EN_US url * teach tools code about --bucket-prefix too

Comment on attachment 8662896 [details] [diff] [review] [buildbotcustom] v1 Review of attachment 8662896 [details] [diff] [review]: ----------------------------------------------------------------- lgtm

Comment on attachment 8662901 [details] [diff] [review] [tools] v1 Review of attachment 8662901 [details] [diff] [review]: ----------------------------------------------------------------- ::: release/patcher-config-bump.pl @@ +17,5 @@ > # we disable this for bug 498273 > # my $RELEASE_CANDIDATE_CHANNELS = ['betatest', 'DisableCompleteJump']; > > my %config; > +my @DEFAULT_PLATFORMS = ('linux', 'linux64', 'macosx64', 'win32'); Feel free to add win64 as well

Comment on attachment 8662901 [details] [diff] [review] [tools] v1 https://hg.mozilla.org/build/tools/rev/b2d4cac21f15

Comment on attachment 8662896 [details] [diff] [review] [buildbotcustom] v1 https://hg.mozilla.org/build/buildbotcustom/rev/300082f07673 https://hg.mozilla.org/build/buildbotcustom/rev/1a685a2b7206

A few staging changes that got lost along the way, and busted the 42.0b5 updates builder with mock_mozilla -r mozilla-centos6-x86_64 --cwd /builds/slave/rel-m-beta-fx_beta_upds-000000/. --unpriv --shell '/usr/bin/env HG_SHARE_BA... perl tools/release/patcher-config-bump.pl -p firefox -r Firefox -v 42.0b5 -a 42.0 -o 42.0b4 -b 1 -c tools/release/patcher-configs/mozBeta-branch-patcher2.cfg -f ftp.mozilla.org -d download.mozilla.org -l shipped-locales --partial-version 42.0b4 --partial-version 42.0b2 --partial-version 42.0b3 --platform linux --platform linux64 --platform macosx64 --platform win32 --platform win64' in dir /builds/slave/rel-m-beta-fx_beta_upds-000000/. (timeout 1200 secs) INFO: mock_mozilla.py version 1.0.3 starting... State Changed: init plugins INFO: selinux disabled State Changed: start State Changed: lock buildroot State Changed: shell Global symbol "$stagingServer" requires explicit package name at tools/release/patcher-config-bump.pl line 307. Global symbol "$stagingServer" requires explicit package name at tools/release/patcher-config-bump.pl line 368. Execution of tools/release/patcher-config-bump.pl aborted due to compilation errors. State Changed: unlock buildroot Landed this fix https://hg.mozilla.org/build/tools/rev/131d55ea931d And this for the review comment about win64 I missed, https://hg.mozilla.org/build/tools/rev/1e67d3d9c321

We got this for 42.0b5 build1 in http://hg.mozilla.org/build/tools/rev/f83dced8be55: + <42.0b5> + checksumsurl http://ftp.mozilla.org/pub/mozilla.org/firefox/candidates/42.0b5-candidates/build1build1/%platform%/%locale%/firefox-42.0b5.checksums + completemarurl http://ftp.mozilla.org/pub/mozilla.org/firefox/candidates/42.0b5-candidates/build1build1/update/%platform%/%locale%/firefox-42.0b5.complete.mar This fixes up the 'build1build1' for correctness's sake, even though I don't think we use these for anything.

Comment on attachment 8671767 [details] [diff] [review] [tools] Fix urls in release block https://hg.mozilla.org/build/tools/rev/323898b72cf8