Closed
Bug 1173459
Opened 9 years ago
Closed 9 years ago
Stop generating partial MAR files and publishing complete MARs to balrog as a part of nightly automation
Categories
(Release Engineering :: General, defect)
Release Engineering
General
Tracking
(firefox42 fixed, firefox43 fixed)
RESOLVED
FIXED
People
(Reporter: rail, Assigned: rail)
References
Details
Attachments
(1 file, 2 obsolete files)
(deleted),
patch
|
mshal
:
review+
Sylvestre
:
approval-mozilla-aurora+
rail
:
checked-in+
|
Details | Diff | Splinter Review |
because we can use Funsize!
Comment 1•9 years ago
|
||
I got here via bug 874885, and I have no idea what funsize is. I don't see any link to any kind of documentation either. Rail, can you please shed some more information on it? What does it mean for updates of nightly builds, and details as included in pulse notifications?
Flags: needinfo?(rail)
Assignee | ||
Comment 2•9 years ago
|
||
We have a wiki page! :) https://wiki.mozilla.org/ReleaseEngineering/Funsize
Flags: needinfo?(rail)
Assignee | ||
Comment 3•9 years ago
|
||
It looks like we can get rid of MOZ_AUTOMATION_UPDATE_PACKAGING which controls partial updates only. We are not ready to turn this of, but it'd be better to be ready in advance.
Attachment #8634217 -
Flags: feedback?(mshal)
Comment 4•9 years ago
|
||
What am I looking at here? It's just a list of files but no diff.
Also just to clear up any confusion, the MOZ_AUTOMATION_UPDATE_PACKAGING symbol also controls whether or not the automation/build target runs 'make update-packaging'. IOW, it drives both completes and partials. To turn off just partial generation, we'll just want to delete the $(if) statement from this line:
https://dxr.mozilla.org/mozilla-central/source/tools/update-packaging/Makefile.in#34
and the automation-partial-patch target:
https://dxr.mozilla.org/mozilla-central/source/tools/update-packaging/Makefile.in#81
(and I guess the partial-patch target if it's now unused?)
Flags: needinfo?(rail)
Assignee | ||
Comment 5•9 years ago
|
||
haha, wrong command :) let me attach a new one
Flags: needinfo?(rail)
Assignee | ||
Comment 6•9 years ago
|
||
This one is better. :)
This is going to be landed whenever we decide to switch to funsize.
Attachment #8634217 -
Attachment is obsolete: true
Attachment #8634217 -
Flags: feedback?(mshal)
Attachment #8634232 -
Flags: review?(mshal)
Comment 7•9 years ago
|
||
Comment on attachment 8634232 [details] [diff] [review]
disable-partials.diff
>diff --git a/build/macosx/mozconfig.common b/build/macosx/mozconfig.common
>--- a/build/macosx/mozconfig.common
>+++ b/build/macosx/mozconfig.common
>@@ -2,17 +2,16 @@
> # License, v. 2.0. If a copy of the MPL was not distributed with this
> # file, You can obtain one at http://mozilla.org/MPL/2.0/.
>
> MOZ_AUTOMATION_L10N_CHECK=0
>
> if [ "x$IS_NIGHTLY" = "xyes" ]; then
> # Some nightlies (eg: Mulet) don't want these set.
> MOZ_AUTOMATION_UPLOAD_SYMBOLS=${MOZ_AUTOMATION_UPLOAD_SYMBOLS-1}
>- MOZ_AUTOMATION_UPDATE_PACKAGING=${MOZ_AUTOMATION_UPDATE_PACKAGING-1}
I think you left this in accidentally? I don't see why we'd want to remove this for mac and not linux/windows. (I think we still want it there for all 3 to get the completes built)
And does funsize use the partial-patch target or can we get rid of that too?
Attachment #8634232 -
Flags: review?(mshal) → feedback+
Assignee | ||
Comment 8•9 years ago
|
||
d'oh, that was a leftover from the previous patch.
Funsize doesn't use any make targets, but we probably should keep that target around - I'm not sure it it's still used by release automation. Also it's a good example on how to create partials.
Thanks for the f+, I'll prep a patch whenever we are ready to switch.
Comment 9•9 years ago
|
||
Sounds good. The rest of it looks awesome! I love patches that just delete ugly code.
Assignee | ||
Comment 10•9 years ago
|
||
Attachment #8634232 -
Attachment is obsolete: true
Attachment #8646908 -
Flags: review?(mshal)
Comment 11•9 years ago
|
||
Comment on attachment 8646908 [details] [diff] [review]
disable_partials.diff
\o/
Attachment #8646908 -
Flags: review?(mshal) → review+
Comment 12•9 years ago
|
||
Assignee | ||
Comment 13•9 years ago
|
||
Comment on attachment 8646908 [details] [diff] [review]
disable_partials.diff
https://hg.mozilla.org/integration/mozilla-inbound/rev/f106756af2a4
Attachment #8646908 -
Flags: checked-in+
Assignee | ||
Comment 15•9 years ago
|
||
Comment on attachment 8646908 [details] [diff] [review]
disable_partials.diff
I'd like to uplift this change and stop generating partial updates as a part of the build. Instead they will be generated by Funsize. Even now, we overwrite partials in balrog with data submitted from funsize. This will be a noop from users point of view.
Attachment #8646908 -
Flags: approval-mozilla-aurora?
Assignee | ||
Comment 16•9 years ago
|
||
Comment on attachment 8646908 [details] [diff] [review]
disable_partials.diff
nm for now, looks like l10n is special and may need more love...
Attachment #8646908 -
Flags: approval-mozilla-aurora?
Assignee | ||
Comment 17•9 years ago
|
||
the balrog step fails even though `make upload` uploads the files, but probably doesn't set some variables properly, see http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2015/08/2015-08-15-03-02-08-mozilla-central-l10n/mozilla-central-linux-l10n-nightly-1-unknown-bm77-build1-build13.txt.gz
06:01:24 INFO - #####
06:01:24 INFO - ##### Running submit-to-balrog step.
06:01:24 INFO - #####
06:01:24 INFO - Running main action method: submit_to_balrog
06:01:24 INFO - Reading buildbot build properties...
06:01:24 INFO - Using buildbot properties:
06:01:24 INFO - {
06:01:24 INFO - "properties": {
06:01:24 INFO - "buildnumber": 13,
06:01:24 INFO - "product": "firefox",
06:01:24 INFO - "script_repo_revision": "production",
06:01:24 INFO - "builddir": "mozilla-central-linux-l10n-nightly-1",
06:01:24 INFO - "repository": "",
06:01:24 INFO - "buildername": "Firefox mozilla-central linux l10n nightly-1",
06:01:24 INFO - "buildid": "20150815030208",
06:01:24 INFO - "stage_platform": "linux",
06:01:24 INFO - "basedir": "/builds/slave/m-cen-lx-l10n-ntly-1-000000000",
06:01:24 INFO - "project": "",
06:01:24 INFO - "platform": "linux",
06:01:24 INFO - "master": "http://buildbot-master77.bb.releng.use1.mozilla.com:8001/",
06:01:24 INFO - "slavebuilddir": "m-cen-lx-l10n-ntly-1-000000000",
06:01:24 INFO - "scheduler": "Linux mozilla-central nightly l10n",
06:01:24 INFO - "branch": "mozilla-central",
06:01:24 INFO - "repo_path": "mozilla-central",
06:01:24 INFO - "slavename": "bld-linux64-spot-041",
06:01:24 INFO - "builduid": "1836627e3bec4917a8ba966ee0c9c4cd",
06:01:24 INFO - "revision": "2ddfc9180971"
06:01:24 INFO - },
06:01:24 INFO - "sourcestamp": {
06:01:24 INFO - "repository": "",
06:01:24 INFO - "hasPatch": false,
06:01:24 INFO - "project": "",
06:01:24 INFO - "branch": "mozilla-central",
06:01:24 INFO - "changes": [],
06:01:24 INFO - "revision": "2ddfc9180971"
06:01:24 INFO - }
06:01:24 INFO - }
06:01:24 INFO - Setting buildbot property branch to mozilla-central
06:01:24 INFO - Setting buildbot property appName to Firefox
06:01:24 INFO - Setting buildbot property hashType to sha512
06:01:24 INFO - Setting buildbot property platform to linux
06:01:24 INFO - Setting buildbot property buildid to 20150815030208
06:01:24 INFO - Setting buildbot property appVersion to 43.0a1
06:01:24 FATAL - Couldn't find complete mar url in config or package_urls
06:01:24 FATAL - Running post_fatal callback...
06:01:24 FATAL - Exiting -1
program finished with exit code 255
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Assignee | ||
Comment 18•9 years ago
|
||
looks like it's https://bugzilla.mozilla.org/show_bug.cgi?id=1194950#c1
Assignee | ||
Updated•9 years ago
|
Assignee | ||
Comment 19•9 years ago
|
||
False alarm, not related to the original patch. Let's try to uplift again.
Assignee | ||
Comment 20•9 years ago
|
||
Comment on attachment 8646908 [details] [diff] [review]
disable_partials.diff
I'd like to uplift this change and stop generating partial updates as a part of the build. Instead they will be generated by Funsize. Even now, we overwrite partials in balrog with data submitted from funsize. This will be a noop from users point of view.
Attachment #8646908 -
Flags: approval-mozilla-aurora?
Updated•9 years ago
|
Attachment #8646908 -
Flags: approval-mozilla-aurora? → approval-mozilla-aurora+
Assignee | ||
Comment 21•9 years ago
|
||
Comment on attachment 8646908 [details] [diff] [review]
disable_partials.diff
https://hg.mozilla.org/releases/mozilla-aurora/rev/60302e9a0a6d
Assignee | ||
Comment 22•9 years ago
|
||
All done here. No more partials as a part of builds!
Status: REOPENED → RESOLVED
Closed: 9 years ago → 9 years ago
Resolution: --- → FIXED
Updated•9 years ago
|
status-firefox42:
--- → fixed
Updated•6 years ago
|
Component: General Automation → General
You need to log in
before you can comment on or make changes to this bug.
Description
•