Closed
Bug 812836
Opened 12 years ago
Closed 12 years ago
Set up nightly multilocale B2G desktop builds for dev+l10n testing
Categories
(Release Engineering :: General, defect)
Release Engineering
General
Tracking
(blocking-basecamp:+, b2g18 fixed)
Tracking | Status | |
---|---|---|
b2g18 | --- | fixed |
People
(Reporter: stas, Assigned: bhearsum)
References
Details
Attachments
(15 files, 15 obsolete files)
(deleted),
patch
|
mozilla
:
review+
akeybl
:
approval-mozilla-aurora+
akeybl
:
approval-mozilla-beta+
bhearsum
:
checked-in+
|
Details | Diff | Splinter Review |
(deleted),
patch
|
mozilla
:
review+
bhearsum
:
checked-in+
|
Details | Diff | Splinter Review |
(deleted),
patch
|
mozilla
:
review+
bhearsum
:
checked-in+
|
Details | Diff | Splinter Review |
(deleted),
patch
|
mozilla
:
review+
bhearsum
:
checked-in+
|
Details | Diff | Splinter Review |
(deleted),
patch
|
mozilla
:
review+
bhearsum
:
checked-in+
|
Details | Diff | Splinter Review |
(deleted),
patch
|
mozilla
:
review+
bhearsum
:
checked-in+
|
Details | Diff | Splinter Review |
(deleted),
patch
|
Details | Diff | Splinter Review | |
(deleted),
patch
|
mozilla
:
review+
bhearsum
:
checked-in+
|
Details | Diff | Splinter Review |
(deleted),
patch
|
rail
:
review+
bhearsum
:
checked-in+
|
Details | Diff | Splinter Review |
(deleted),
patch
|
mozilla
:
review+
bhearsum
:
checked-in+
|
Details | Diff | Splinter Review |
(deleted),
patch
|
mozilla
:
review+
bhearsum
:
checked-in+
|
Details | Diff | Splinter Review |
(deleted),
patch
|
mozilla
:
review+
|
Details | Diff | Splinter Review |
(deleted),
patch
|
mozilla
:
review+
bhearsum
:
checked-in+
|
Details | Diff | Splinter Review |
(deleted),
patch
|
mozilla
:
review+
bhearsum
:
checked-in+
|
Details | Diff | Splinter Review |
(deleted),
patch
|
mozilla
:
review+
bhearsum
:
checked-in+
|
Details | Diff | Splinter Review |
This is similar to bug 812835, but for dektop, not Unagi.
I'd like us to start producing special multilocale desktop builds of B2G for localization testing. While bug 812834 is about adding a few locales to our regular desktop builds (in order to allow the testing of locale changing, font rendering etc), this bug is about providing mass-multilocale builds for our localization community.
gaia locales: all Gaia locales (see https://hg.mozilla.org/gaia-l10n/)
gecko locales: all Fennec locales
Assignee | ||
Updated•12 years ago
|
Assignee: nobody → bhearsum
Assignee | ||
Comment 1•12 years ago
|
||
The current nightly desktop builds (the en-US ones) are MercurialBuildFactory based. Here's my rough outline of what we need to do here:
* Adjust those builders to clone the l10n repos before calling 'make -f client.mk build'. Not sure how to implement this yet, because the existing multilocale support we have in MBF is called after the build step.
* Pass something to the build system to indicate that we want a multilocale build. This might be in the mozconfig, it might be an environment variable. Hopefully this can be the name of or symbolic name of one of the languages.json files. This Makefile may have to be adjusted to pass that along to Gaia's build system, depending how bug 812829 is implemented:
https://mxr.mozilla.org/mozilla-central/source/b2g/gaia/Makefile.in#39
* Pass something to the build system to indicate which directory the l10n repos will be in. Again, this may have to be forwarded in b2g/gaia/Makefile.in.
After those things, we should end up with a multilocale desktop build. Still not sure where we're going to alter the existing nightly desktop build for this or add a new one.
Assignee | ||
Comment 2•12 years ago
|
||
This gets us as far as cloning the gaia l10n repos. It depends on an update to the gecko config to add an "l10n" section to it. I haven't figured out how to pass along the gecko config for MBF based builds yet.
Aki, I gave up trying to touch the l10n mixins and put everything I needed here. Without major refactoring I couldn't make it work without adding a completely independent method to the class, which seemed worse than putting it here. We'll likely need LocalesMixin when we figure out the gecko part of the multilocale builds.
This still needs whatever bits that make the Gaia build system do what we want plugged in, we don't know exactly what that will like yet though.
I'm intending to call this script with --checkout-gaia-l10n for this bug.
I'm still not sure if it should be in default actions, since it depends on the optional argument --gaia-languages-file. This could cause headaches/bustage.
Attachment #684184 -
Flags: feedback?(aki)
Comment 3•12 years ago
|
||
Comment on attachment 684184 [details] [diff] [review]
add gaia l10n support to b2g_build.py
>+ if 'gaia_languages_file' in self.config:
>+ if not self.load_gecko_config().get('gaia').get('l10n'):
>+ self.fatal("gaia.l10n is required in the gecko config when --gaia-languages-file is specified.")
That might need to be
if not self.load_gecko_config().get('gaia', {}).get('l10n'):
>+ def checkout_gaia_l10n(self):
>+ l10n_base_dir = self.query_abs_dirs()['l10n_base_dir']
>+ l10n_config = self.load_gecko_config().get('gaia').get('l10n')
Since you're doing this check in __init__() I'd be fine going straight to gecko_config['gaia']['l10n']
Also, to simplify things, I'd be fine with
def checkout_gaia_l10n(self):
if not self.config.get("gaia_languages_file"):
self.info("Skipping checkout_gaia_l10n!")
return
...
Which would make deploying en-US deps versus multilocale nightlies easier.
Or we can modify default_actions for dep-vs-nightly.
>+ root = l10n_config['root']
>+ # urljoin will strip the last part of root if it doesn't end with "/"
>+ if not root.endswith('/'):
>+ root = root + '/'
>+ vcs = l10n_config['vcs']
>+ languages_file = parse_config_file(self.config['gaia_languages_file'])
>+ repos = []
>+ # generate list of repos
>+ for locale in languages_file.keys():
>+ repos.append({
>+ 'repo': urljoin(root, locale),
>+ 'dest': locale,
>+ 'vcs': vcs,
>+ })
>+ # call vcs_clone_repos
>+ self.vcs_checkout_repos(repo_list=repos, parent_dir=l10n_base_dir)
>+ # then we need to pass down options to gaia build system in build()
I think this will work.
Do you only need the l10n repos? (no compare-locales equivalents or anything?) Do you also need to remove any directories, or is that solved by pointing at this new l10n parent_dir?
'dest' is unneeded, as it'll default to the basename of the repo, but is harmless.
Seems like you've gotten the hang of things pretty quickly, as I thought you would :)
Attachment #684184 -
Flags: feedback?(aki) → feedback+
Assignee | ||
Comment 4•12 years ago
|
||
(In reply to Aki Sasaki [:aki] from comment #3)
> Comment on attachment 684184 [details] [diff] [review]
> add gaia l10n support to b2g_build.py
>
> >+ if 'gaia_languages_file' in self.config:
> >+ if not self.load_gecko_config().get('gaia').get('l10n'):
> >+ self.fatal("gaia.l10n is required in the gecko config when --gaia-languages-file is specified.")
>
> That might need to be
>
> if not self.load_gecko_config().get('gaia', {}).get('l10n'):
Good catch...
> Also, to simplify things, I'd be fine with
>
> def checkout_gaia_l10n(self):
> if not self.config.get("gaia_languages_file"):
> self.info("Skipping checkout_gaia_l10n!")
> return
> ...
> Which would make deploying en-US deps versus multilocale nightlies easier.
Looks like we don't have to worry about that actually, per e-mail. I think we can just make checkout-gaia-l10n a default action and be done with it, since we won't be doing any en-US only builds when we're done?
> >+ root = l10n_config['root']
> >+ # urljoin will strip the last part of root if it doesn't end with "/"
> >+ if not root.endswith('/'):
> >+ root = root + '/'
> >+ vcs = l10n_config['vcs']
> >+ languages_file = parse_config_file(self.config['gaia_languages_file'])
> >+ repos = []
> >+ # generate list of repos
> >+ for locale in languages_file.keys():
> >+ repos.append({
> >+ 'repo': urljoin(root, locale),
> >+ 'dest': locale,
> >+ 'vcs': vcs,
> >+ })
> >+ # call vcs_clone_repos
> >+ self.vcs_checkout_repos(repo_list=repos, parent_dir=l10n_base_dir)
> >+ # then we need to pass down options to gaia build system in build()
>
> I think this will work.
> Do you only need the l10n repos? (no compare-locales equivalents or
> anything?)
There's no compare-locales that will work with Gaia, and I don't think there's any plans to have one at the moment.
> Do you also need to remove any directories, or is that solved by
> pointing at this new l10n parent_dir?
Do you mean remove any existing locale directories that exist? If so, I don't think we do because hgtool will take care of that if necessary.
> 'dest' is unneeded, as it'll default to the basename of the repo, but is
> harmless.
Yeah, I think I'll just leave it in to be safe.
> Seems like you've gotten the hang of things pretty quickly, as I thought you
> would :)
Not sure if you're complimenting me or yourself :P. But thanks!
Assignee | ||
Comment 5•12 years ago
|
||
I assume we're going to need to backport this to aurora and beta, too. There's no reason this can't land before the build system parts are right.
Attachment #684470 -
Flags: review?(aki)
Assignee | ||
Comment 6•12 years ago
|
||
With this + the other ones I'm at the point where unagi jobs are cloning the l10n repos. Still waiting to hook into the build system bits that are tracked in bug 812829.
Changes since the last patch:
* Stop looking for gaia.l10n in gecko config as part of the pre lock check. We can't do this because the gecko config isn't available until checkout-gecko runs.
* Skip checkout-gaia-l10n if gaia_languages_file isn't set.
* Make gaia languages file an absolute path when reading it, so that it always works.
Attachment #684184 -
Attachment is obsolete: true
Attachment #684472 -
Flags: feedback?(aki)
Assignee | ||
Comment 7•12 years ago
|
||
D'oh, I got this bug mixed up with the unagi ones. The posted patches are relevant, but I still need to test my buildbotcustom patch -- ignore the parts about testing the unagi build.
Assignee | ||
Comment 8•12 years ago
|
||
Filename may change, this is the file that exists right now though.
Attachment #684475 -
Flags: review?(aki)
Assignee | ||
Comment 9•12 years ago
|
||
Comment on attachment 684472 [details] [diff] [review]
address comments, fix bugs
Had to move gaia l10n cloning to a mixin, couldn't use this script for these builds because we don't have a gecko config. New patch coming in a bit.
Attachment #684472 -
Attachment is obsolete: true
Attachment #684472 -
Flags: feedback?(aki)
Assignee | ||
Comment 10•12 years ago
|
||
Changes since the first patch:
* Put gaia l10n stuff in a mixin. Note that I'm passing parameters to it instead of reading the config, because I'm going to need LocalesMixin at some point too and I don't want config parameters to conflict.
* Create a new script for cloning gaia repos. It's tiny and basically just calls the mixin.
I don't have this fully tested yet because hgtool isn't working for some reason.
Attachment #684515 -
Flags: feedback?(aki)
Assignee | ||
Comment 11•12 years ago
|
||
* Makes mozharness cloning independent of multilocale settings
* Adds gaiaLanguagesFile/Script/L10nRoot, all forwarded along to the cloning script
Attachment #684521 -
Flags: feedback?(aki)
Updated•12 years ago
|
Attachment #684470 -
Flags: review?(aki) → review+
Updated•12 years ago
|
Attachment #684475 -
Flags: review?(aki) → review+
Comment 12•12 years ago
|
||
Comment on attachment 684515 [details] [diff] [review]
use mixin for gaia locale stuff
I can help you tie this into LocalesMixin later if need be.
I think hgtool.py is deployed to all mock slaves; where were you testing?
You can grab http://people.mozilla.org/~catlee/hgtool.py if you want to test elsewhere.
I'm basically eyeballing this; let me know if there are other things that you need help debugging once you get hgtool working.
Attachment #684515 -
Flags: feedback?(aki) → feedback+
Comment 13•12 years ago
|
||
Comment on attachment 684521 [details] [diff] [review]
buildbotcustom integration
When I test-masters.sh this patch with the buildbot-configs patch, I'm getting
TypeError: type object got multiple values for keyword argument 'mozharnessTag'
>+ self.addStep(MockCommand(
>+ name='clone_gaia_l10n_repos',
This is probably why hgtool is failing. It's in /usr/local/bin on the host, but aiui isn't copied into the mock environment.
We either need to copy it in, or do the clone from the host without mock.
Aiui, the latter is known to work, but you'll have to prepend the /build/mock_mozilla/... to the target path; I'm not sure if the former has ever been successfully [or unsuccessfully] tested.
Attachment #684521 -
Flags: feedback?(aki) → feedback+
Comment 14•12 years ago
|
||
(In reply to Ben Hearsum [:bhearsum] from comment #4)
> (In reply to Aki Sasaki [:aki] from comment #3)
...
> > I think this will work.
> > Do you only need the l10n repos? (no compare-locales equivalents or
> > anything?)
>
> There's no compare-locales that will work with Gaia, and I don't think
> there's any plans to have one at the moment.
The compare-dirs command is part of the compare-locales python package, and the gaia-l10n setup is made for that command to work.
What I don't know is if and what runtime error handling we have today, or will have during the build process, which may or may not make l10n-merge not needed. Yay if we knew what we're shipping. [sarcasm-sign]
Anyway, if we end up needing l10n-merge, the code is there and exists. As we're not using JarMaker, the build logic that picks up files from merge/l10n/en-US would need to live "somewhere else", see above.
Assignee | ||
Comment 15•12 years ago
|
||
(In reply to Aki Sasaki [:aki] from comment #13)
> Comment on attachment 684521 [details] [diff] [review]
> buildbotcustom integration
>
> When I test-masters.sh this patch with the buildbot-configs patch, I'm
> getting
>
> TypeError: type object got multiple values for keyword argument
> 'mozharnessTag'
Damn. I was testing with most builders disabled. I guess mozharnessTag gets conditionally added to the args somewhere. I'll fix that up.
> >+ self.addStep(MockCommand(
> >+ name='clone_gaia_l10n_repos',
>
> This is probably why hgtool is failing. It's in /usr/local/bin on the host,
> but aiui isn't copied into the mock environment.
> We either need to copy it in, or do the clone from the host without mock.
> Aiui, the latter is known to work, but you'll have to prepend the
> /build/mock_mozilla/... to the target path; I'm not sure if the former has
> ever been successfully [or unsuccessfully] tested.
We have a list of mock copy in files somewhere...I probably just need to make sure that gets set.
Assignee | ||
Comment 16•12 years ago
|
||
(In reply to Axel Hecht [:Pike] from comment #14)
> (In reply to Ben Hearsum [:bhearsum] from comment #4)
> > (In reply to Aki Sasaki [:aki] from comment #3)
> ...
> > > I think this will work.
> > > Do you only need the l10n repos? (no compare-locales equivalents or
> > > anything?)
> >
> > There's no compare-locales that will work with Gaia, and I don't think
> > there's any plans to have one at the moment.
>
> The compare-dirs command is part of the compare-locales python package, and
> the gaia-l10n setup is made for that command to work.
>
> What I don't know is if and what runtime error handling we have today, or
> will have during the build process, which may or may not make l10n-merge not
> needed. Yay if we knew what we're shipping. [sarcasm-sign]
>
> Anyway, if we end up needing l10n-merge, the code is there and exists. As
> we're not using JarMaker, the build logic that picks up files from
> merge/l10n/en-US would need to live "somewhere else", see above.
I'll send some mail to try and figure these details out. I'm going to work on the assumption that we're not doing anything with compare-locales for now, though.
Assignee | ||
Comment 17•12 years ago
|
||
(In reply to Axel Hecht [:Pike] from comment #14)
> (In reply to Ben Hearsum [:bhearsum] from comment #4)
> > (In reply to Aki Sasaki [:aki] from comment #3)
> ...
> > > I think this will work.
> > > Do you only need the l10n repos? (no compare-locales equivalents or
> > > anything?)
> >
> > There's no compare-locales that will work with Gaia, and I don't think
> > there's any plans to have one at the moment.
>
> The compare-dirs command is part of the compare-locales python package, and
> the gaia-l10n setup is made for that command to work.
>
> What I don't know is if and what runtime error handling we have today, or
> will have during the build process, which may or may not make l10n-merge not
> needed. Yay if we knew what we're shipping. [sarcasm-sign]
>
> Anyway, if we end up needing l10n-merge, the code is there and exists. As
> we're not using JarMaker, the build logic that picks up files from
> merge/l10n/en-US would need to live "somewhere else", see above.
Oops, sorry! I thought this was an Aki comment! So to summarize this: we should be running "compare-dirs" before generating a Gaia multilocale profile? If so, can you provide a sample invocation?
Comment 18•12 years ago
|
||
I guess we need stas for details here, I don't think I have the latest emails.
My latest info is that we have no invocation on either the gecko nor the gaia side.
Flags: needinfo?(stas)
Assignee | ||
Comment 19•12 years ago
|
||
Same as the previous patch, but with new builds that use different language files. This patch will end up resolving bug 812834 too because it modifies the existing builders, too. I'll probably dupe that bug over later.
Attachment #684475 -
Attachment is obsolete: true
Attachment #684729 -
Flags: review?(aki)
Assignee | ||
Comment 20•12 years ago
|
||
A few things new in this version:
* Fix errors about duplicate arguments. I removed mozharnessRepoPath and Tag from multiargs because they will always get added now.
* Add some sanity checks when gaiaLanguagesFile is set
* Update the env to pass LOCALE_BASEDIR when gaiaLanguagesFile is set
* Pass --gaia-locale-base-dir to clone script, because otherwise we have to guess or assume where it is.
Attachment #684521 -
Attachment is obsolete: true
Attachment #684730 -
Flags: review?(aki)
Assignee | ||
Comment 21•12 years ago
|
||
New in this version:
* Set LOCALE_BASEDIR in b2g_build.py
* Allow locale base dir to be passed to clone script.
Still to do:
* Set languages file when we know which env var will be used for that.
Attachment #684515 -
Attachment is obsolete: true
Attachment #684731 -
Flags: feedback?(aki)
Assignee | ||
Comment 22•12 years ago
|
||
Comment on attachment 684470 [details] [diff] [review]
add gaia l10n info to gecko config for unagi
Landed this on mozilla-central: https://hg.mozilla.org/mozilla-central/rev/4d832dcd0be9
Attachment #684470 -
Flags: checked-in+
Assignee | ||
Comment 23•12 years ago
|
||
Comment on attachment 684470 [details] [diff] [review]
add gaia l10n info to gecko config for unagi
[Approval Request Comment]
Bug caused by (feature/regressing bug #): N/A
User impact if declined: Can't setup required localizer and dev multilocale builds on aurora/beta
Testing completed (on m-c, etc.): None so far. This patch is a no-op for current builds. Required to support builds requested in this bug and bugs 812833, 812834, and 812835.
Risk to taking this patch (and alternatives if risky): N/A
String or UUID changes made by this patch: None
Attachment #684470 -
Flags: approval-mozilla-beta?
Attachment #684470 -
Flags: approval-mozilla-aurora?
Comment 24•12 years ago
|
||
Comment on attachment 684729 [details] [diff] [review]
add builders for localizer builds
I was wondering whether these would be dep or nightly, and what the NOT-IN-BB-CONF is, but after digging deeper this looks correct.
For hgtool, the mock copyin files may be here, e.g.
+ 'mock_copyin_files': [
+ ('/home/cltbld/.hgrc', '/builds/.hgrc'),
+ ('/home/cltbld/.ssh', '/home/mock_mozilla/.ssh'),
+ ],
Not sure if there's also an in-tree mock config.
Attachment #684729 -
Flags: review?(aki) → review+
Comment 25•12 years ago
|
||
Comment on attachment 684730 [details] [diff] [review]
updated buildbotcustom patch
Looks like this fixed the test-masters.sh issue.
Attachment #684730 -
Flags: review?(aki) → review+
Comment 26•12 years ago
|
||
Comment on attachment 684731 [details] [diff] [review]
latest mozharness patch
This looks right. It does look like you'll need --target no matter what to read the gecko config, so that's a good change.
Attachment #684731 -
Flags: feedback?(aki) → feedback+
Assignee | ||
Comment 27•12 years ago
|
||
(In reply to Aki Sasaki [:aki] from comment #24)
> Comment on attachment 684729 [details] [diff] [review]
> add builders for localizer builds
>
> I was wondering whether these would be dep or nightly, and what the
> NOT-IN-BB-CONF is, but after digging deeper this looks correct.
NOT-IN-BB-CONF is there because misc.py is stupid and requires that variable set =\.
> For hgtool, the mock copyin files may be here, e.g.
>
> + 'mock_copyin_files': [
> + ('/home/cltbld/.hgrc', '/builds/.hgrc'),
> + ('/home/cltbld/.ssh', '/home/mock_mozilla/.ssh'),
> + ],
>
> Not sure if there's also an in-tree mock config.
It's not useful to us for these jobs even there is AFAIK. We can't read in files from the slave to feed to any of the mock prep steps.
Comment 28•12 years ago
|
||
Comment on attachment 684470 [details] [diff] [review]
add gaia l10n info to gecko config for unagi
no-op for current builds, required for B2G multilocale builds, approving for branches.
Attachment #684470 -
Flags: approval-mozilla-beta?
Attachment #684470 -
Flags: approval-mozilla-beta+
Attachment #684470 -
Flags: approval-mozilla-aurora?
Attachment #684470 -
Flags: approval-mozilla-aurora+
Assignee | ||
Comment 29•12 years ago
|
||
Comment on attachment 684470 [details] [diff] [review]
add gaia l10n info to gecko config for unagi
Landed on beta: https://hg.mozilla.org/releases/mozilla-beta/rev/231dcf277559
Haven't landed on aurora yet because it's closed at the moment.
Assignee | ||
Comment 30•12 years ago
|
||
Comment on attachment 684470 [details] [diff] [review]
add gaia l10n info to gecko config for unagi
This is NPOTB so I pushed anyways: https://hg.mozilla.org/releases/mozilla-aurora/rev/89b21c061697
Updated•12 years ago
|
blocking-basecamp: --- → +
Target Milestone: --- → B2G C2 (20nov-10dec)
Assignee | ||
Comment 31•12 years ago
|
||
Only thing new here is adding the env vars to trigger the right multilocale behaviour in the build system.
Attachment #684731 -
Attachment is obsolete: true
Attachment #686570 -
Flags: review?(aki)
Assignee | ||
Comment 32•12 years ago
|
||
New since the last patch:
* Add _localizer to the supportedPlatforms list. I hate this, but doing a variant based on the same platform is worse.
* Be a bit more liberal with Linux package names (because we use MOZ_PKG_SPECIAL to ensure uniqueness for localizer builds)
* Set GAIA_LANGUAGES_FILE so that clone_gaia_locales.py works.
Attachment #684730 -
Attachment is obsolete: true
Attachment #686574 -
Flags: review?(aki)
Assignee | ||
Comment 33•12 years ago
|
||
New in this patch:
* Add localizer builders to the default set of platforms to enable them everywhere.
* Change gaia_languages_file for existing desktop builders to point at languages-dev.json, because that will be the file with the 6 locale set.
* Change stage_platform for existing mac/windows desktop builders to include _gecko, like Linux. As I understand it, this only affects the directory names in tinderbox-builds, but it seems like we should be consistent here.
* Set MOZ_PKG_SPECIAL to "localizer" for localizer builds, to make sure we get unique filenames. Without this, localizer and non-localizer builds will stomp on each other in latest-*
* Pass --gaia-languages-file to panda/unagi/otoro mozharness scripts.
Assuming the builds that I gave devs/QA to look at last night don't have any issues, I intend to land these patches today.
Attachment #686584 -
Flags: review?(aki)
Updated•12 years ago
|
Attachment #686570 -
Flags: review?(aki) → review+
Updated•12 years ago
|
Attachment #686574 -
Flags: review?(aki) → review+
Comment 34•12 years ago
|
||
Comment on attachment 686584 [details] [diff] [review]
enable localizer builders; a few other fixes
We're changing upload locations, which is probably fine.
Might be good to announce the ftp location change when you announce that multilocale has landed \o/
Attachment #686584 -
Flags: review?(aki) → review+
Assignee | ||
Updated•12 years ago
|
Attachment #684729 -
Attachment is obsolete: true
Assignee | ||
Updated•12 years ago
|
Attachment #686570 -
Flags: checked-in+
Assignee | ||
Updated•12 years ago
|
Attachment #686574 -
Flags: checked-in+
Assignee | ||
Updated•12 years ago
|
Attachment #686584 -
Flags: checked-in+
Assignee | ||
Updated•12 years ago
|
Flags: needinfo?(stas)
Assignee | ||
Comment 35•12 years ago
|
||
I landed the mozharness patch slightly ahead of the buildbot parts and ended up hitting a KeyError in some places. With the buildbot parts landed I don't think we'll hit it anymore, but this patch should make --gaia-languages-file truly optional.
Attachment #687074 -
Flags: review?(aki)
Assignee | ||
Comment 36•12 years ago
|
||
Builder names aren't great on the localizer builds -- need to remove "localizer" from the base name since I added it to the platform name.
Assignee | ||
Comment 37•12 years ago
|
||
May need to switch the environments to centos6, given bug 816793.
Also, I had to back out changes to unagi/panda/otoro because Mozharness was errantly throwing Automation errors after failing to find bundles for the locales:
07:28:04 INFO - #####
07:28:04 INFO - ##### Running checkout-gaia-l10n step.
07:28:04 INFO - #####
07:28:04 INFO - mkdir: /builds/slave/b2g-m-cen-panda-dep/build/gaia-l10n
07:28:04 INFO - Changing directory to /builds/slave/b2g-m-cen-panda-dep/build/gaia-l10n.
07:28:04 INFO - Running command: ['hgtool.py', '--mirror', u'http://hg-internal.dmz.scl3.mozilla.com/gaia-l10n/fr', '--bundle', u'http://ftp.mozilla.org/pub/mozilla.org/firefox/bundles/fr.hg', u'http://hg.mozilla.org/gaia-l10n/fr', u'/builds/slave/b2g-m-cen-panda-dep/build/gaia-l10n/fr']
07:28:04 INFO - Copy/paste: hgtool.py --mirror http://hg-internal.dmz.scl3.mozilla.com/gaia-l10n/fr --bundle http://ftp.mozilla.org/pub/mozilla.org/firefox/bundles/fr.hg http://hg.mozilla.org/gaia-l10n/fr /builds/slave/b2g-m-cen-panda-dep/build/gaia-l10n/fr
07:28:04 INFO - Checking if share extension works
07:28:04 INFO - command: START
07:28:04 INFO - command: hg help share
07:28:04 INFO - command: cwd: /builds/slave/b2g-m-cen-panda-dep/build/gaia-l10n
07:28:04 INFO - command: output:
07:28:04 INFO - command: END (0.37 elapsed)
07:28:04 INFO - Updating shared repo
07:28:04 INFO - Attempting to initialize clone with bundles
07:28:04 INFO - command: START
07:28:04 INFO - command: hg init /builds/hg-shared/gaia-l10n/fr
07:28:04 INFO - command: cwd: /builds/slave/b2g-m-cen-panda-dep/build/gaia-l10n
07:28:04 INFO - command: output:
07:28:05 INFO - command: END (0.37s elapsed)
07:28:05 INFO - Trying to use bundle http://ftp.mozilla.org/pub/mozilla.org/firefox/bundles/fr.hg
07:28:05 INFO - command: START
07:28:05 INFO - command: hg unbundle http://ftp.mozilla.org/pub/mozilla.org/firefox/bundles/fr.hg
07:28:05 INFO - command: cwd: /builds/hg-shared/gaia-l10n/fr
07:28:05 INFO - command: output:
07:28:06 ERROR - abort: HTTP Error 404: Not Found
07:28:06 ERROR - Automation Error: hg not responding
07:28:06 INFO - command: ERROR
07:28:06 INFO - Traceback (most recent call last):
07:28:06 INFO - File "<string>", line 42, in run_cmd
07:28:06 INFO - File "/usr/lib64/python2.6/subprocess.py", line 502, in check_call
07:28:06 INFO - raise CalledProcessError(retcode, cmd)
07:28:06 INFO - CalledProcessError: Command '['hg', 'unbundle', 'http://ftp.mozilla.org/pub/mozilla.org/firefox/bundles/fr.hg']' returned non-zero exit status 255
07:28:06 INFO - command: END (0.73s elapsed)
07:28:06 INFO - Problem unbundling/pulling from http://ftp.mozilla.org/pub/mozilla.org/firefox/bundles/fr.hg
07:28:06 INFO - Traceback (most recent call last):
07:28:06 INFO - File "<string>", line 142, in clone
07:28:06 INFO - File "<string>", line 505, in unbundle
07:28:06 INFO - File "<string>", line 42, in run_cmd
07:28:06 INFO - File "/usr/lib64/python2.6/subprocess.py", line 502, in check_call
07:28:06 INFO - raise CalledProcessError(retcode, cmd)
07:28:06 INFO - CalledProcessError: Command '['hg', 'unbundle', 'http://ftp.mozilla.org/pub/mozilla.org/firefox/bundles/fr.hg']' returned non-zero exit status 255
07:28:06 INFO - Using bundles failed; falling back to clone
07:28:06 INFO - Attempting to clone from mirrors
07:28:06 INFO - Cloning from http://hg-internal.dmz.scl3.mozilla.com/gaia-l10n/fr
07:28:06 INFO - command: START
07:28:06 INFO - command: hg clone -U http://hg-internal.dmz.scl3.mozilla.com/gaia-l10n/fr /builds/hg-shared/gaia-l10n/fr
07:28:06 INFO - command: cwd: /builds/slave/b2g-m-cen-panda-dep/build/gaia-l10n
07:28:06 INFO - command: output:
07:28:09 INFO - requesting all changes
07:28:09 INFO - adding changesets
07:28:09 INFO - adding manifests
07:28:09 INFO - adding file changes
07:28:09 INFO - added 34 changesets with 120 changes to 24 files
07:28:09 INFO - command: END (3.61s elapsed)
07:28:09 INFO - Trying to share /builds/hg-shared/gaia-l10n/fr to /builds/slave/b2g-m-cen-panda-dep/build/gaia-l10n/fr
07:28:09 INFO - command: START
07:28:09 INFO - command: hg share -U /builds/hg-shared/gaia-l10n/fr /builds/slave/b2g-m-cen-panda-dep/build/gaia-l10n/fr
07:28:09 INFO - command: cwd: /builds/slave/b2g-m-cen-panda-dep/build/gaia-l10n
07:28:09 INFO - command: output:
07:28:10 INFO - command: END (0.38s elapsed)
07:28:10 INFO - command: START
07:28:10 INFO - command: hg branch
07:28:10 INFO - command: cwd: /builds/slave/b2g-m-cen-panda-dep/build/gaia-l10n/fr
07:28:10 INFO - command: output:
07:28:10 INFO - default
07:28:10 INFO - command: END (0.39 elapsed)
07:28:10 INFO - command: START
07:28:10 INFO - command: hg update -C
07:28:10 INFO - command: cwd: /builds/slave/b2g-m-cen-panda-dep/build/gaia-l10n/fr
07:28:10 INFO - command: output:
07:28:10 INFO - 24 files updated, 0 files merged, 0 files removed, 0 files unresolved
07:28:10 INFO - command: END (0.48s elapsed)
07:28:10 INFO - command: START
07:28:10 INFO - command: hg parent --template {node|short}
07:28:10 INFO - command: cwd: /builds/slave/b2g-m-cen-panda-dep/build/gaia-l10n/fr
07:28:10 INFO - command: output:
07:28:11 INFO - f2bac708698c
07:28:11 INFO - command: END (0.43 elapsed)
07:28:11 INFO - Got revision f2bac708698c
Assignee | ||
Comment 38•12 years ago
|
||
I think https://bugzilla.mozilla.org/show_bug.cgi?id=816977 is tracking the automation error issue.
Assignee | ||
Comment 39•12 years ago
|
||
Comment on attachment 687074 [details] [diff] [review]
don't fail when gaia_languages_file isn't set
Had to land this to disable otoro/panda/unagi multilocale.
Attachment #687074 -
Flags: checked-in+
Assignee | ||
Comment 40•12 years ago
|
||
This landing was....complicated, so I want to summarize where we're at now.
* Multilocale gaia is enabled for b2g desktop builds. The builds are currently failing due to a compile failed introduced by bug 809106.
* Multilocale gaia for unagi/otoro/panda was enabled, and then had to be backed out because Mozharness throwing spurious errors about "failed" hg clones. Details of that are in bug 816977.
Still to do here:
* Fix localizer build names to not say "localizer" twice.
* Verify that desktop builds actually work after bug 809106 is fixed.
* Re-enable multilocale gaia for otoro/unagi/panda after bug 816977 is fixed or otherwise worked around.
And of course, we still have to enable multilocale gecko, but that still depends on build system work in bug 812830.
Depends on: 816977
Assignee | ||
Comment 41•12 years ago
|
||
Attachment #687144 -
Flags: review?(aki)
Updated•12 years ago
|
Attachment #687074 -
Flags: review?(aki) → review+
Updated•12 years ago
|
Attachment #687144 -
Flags: review?(aki) → review+
Assignee | ||
Comment 42•12 years ago
|
||
(In reply to Ben Hearsum [:bhearsum] from comment #40)
> * Multilocale gaia for unagi/otoro/panda was enabled, and then had to be
> backed out because Mozharness throwing spurious errors about "failed" hg
> clones. Details of that are in bug 816977.
I think this is wrong, actually. I just noticed this in the device build logs while looking at them with Aki:
INFO - cp: cannot stat
`/builds/slave/b2g-m-cen-otoro-dep/build/gaia/profile/user.js': No
such file or directory
Maybe needed a clobber after enabling multilocale?
Assignee | ||
Updated•12 years ago
|
Attachment #687144 -
Flags: checked-in+
Assignee | ||
Comment 43•12 years ago
|
||
Per e-mail, when we turn back on the device builds, we don't want otoro yet because no one has been able to flash one of the otoro multilocale images yet without bricking their device.
Assignee | ||
Comment 44•12 years ago
|
||
(In reply to Ben Hearsum [:bhearsum] from comment #42)
> (In reply to Ben Hearsum [:bhearsum] from comment #40)
> > * Multilocale gaia for unagi/otoro/panda was enabled, and then had to be
> > backed out because Mozharness throwing spurious errors about "failed" hg
> > clones. Details of that are in bug 816977.
>
> I think this is wrong, actually. I just noticed this in the device build
> logs while looking at them with Aki:
> INFO - cp: cannot stat
>
> `/builds/slave/b2g-m-cen-otoro-dep/build/gaia/profile/user.js': No
> such file or directory
>
> Maybe needed a clobber after enabling multilocale?
I just tested this theory and it doesn't hold -- even after a clobber I can't get past this error. I'm betting that something landed since I last tested that broke this. Still investigating, though.
Assignee | ||
Comment 45•12 years ago
|
||
(In reply to Ben Hearsum [:bhearsum] from comment #44)
> (In reply to Ben Hearsum [:bhearsum] from comment #42)
> > (In reply to Ben Hearsum [:bhearsum] from comment #40)
> > > * Multilocale gaia for unagi/otoro/panda was enabled, and then had to be
> > > backed out because Mozharness throwing spurious errors about "failed" hg
> > > clones. Details of that are in bug 816977.
> >
> > I think this is wrong, actually. I just noticed this in the device build
> > logs while looking at them with Aki:
> > INFO - cp: cannot stat
> >
> > `/builds/slave/b2g-m-cen-otoro-dep/build/gaia/profile/user.js': No
> > such file or directory
> >
> > Maybe needed a clobber after enabling multilocale?
>
> I just tested this theory and it doesn't hold -- even after a clobber I
> can't get past this error. I'm betting that something landed since I last
> tested that broke this. Still investigating, though.
Looking in the build directory, I can confirm that that file doesn't exist:
bash-4.1# pwd
/builds/slave/b2g-m-cen-otoro-dep/build/gaia
bash-4.1# ls profile
settings.json webapps
Stas, I've attached a full build log. Any idea what's happening here?
Attachment #687268 -
Flags: review?(stas)
Assignee | ||
Comment 46•12 years ago
|
||
With bug 817071 fixed, we managed to get some OS X desktop builds. Windows ones still seem to be failing with a non-code related issue, though:
/bin/sh: e:/builds/moz2_slave/m-cen-w32-gecko-localizer-ntly/build/obj-firefox/_virtualenv/Scripts/python.exe: Bad file number
No idea why this is, will have to look into this next week.
Depends on: 817071
Assignee | ||
Comment 47•12 years ago
|
||
(In reply to Ben Hearsum [:bhearsum] from comment #46)
> With bug 817071 fixed, we managed to get some OS X desktop builds. Windows
> ones still seem to be failing with a non-code related issue, though:
> /bin/sh:
> e:/builds/moz2_slave/m-cen-w32-gecko-localizer-ntly/build/obj-firefox/
> _virtualenv/Scripts/python.exe: Bad file number
>
> No idea why this is, will have to look into this next week.
This might be a specific machine issue rather than a general one. The other win32 desktop build succeeded: https://ftp.mozilla.org/pub/mozilla.org/b2g/nightly/latest-mozilla-central/b2g-20.0a1.en-US.win32.zip.
Reporter | ||
Comment 49•12 years ago
|
||
Comment on attachment 687268 [details] [diff] [review]
output from clobber build of otoro w/ multilocale enabled
I looked at the log but I don't have any good ideas about why the build might be failing.
Maybe the 'preferences' target fails to find the files defined in EXTENDED_PREF_FILES ? Can you try to add some debugging output somewhere near line 333?
https://github.com/mozilla-b2g/gaia/blob/92c343c3b376ec8b6bee3c55c8bbb6fb7acd4d25/Makefile#L333
Attachment #687268 -
Flags: review?(stas)
Assignee | ||
Comment 50•12 years ago
|
||
(In reply to Ben Hearsum [:bhearsum] from comment #47)
> (In reply to Ben Hearsum [:bhearsum] from comment #46)
> > With bug 817071 fixed, we managed to get some OS X desktop builds. Windows
> > ones still seem to be failing with a non-code related issue, though:
> > /bin/sh:
> > e:/builds/moz2_slave/m-cen-w32-gecko-localizer-ntly/build/obj-firefox/
> > _virtualenv/Scripts/python.exe: Bad file number
> >
> > No idea why this is, will have to look into this next week.
>
> This might be a specific machine issue rather than a general one. The other
> win32 desktop build succeeded:
> https://ftp.mozilla.org/pub/mozilla.org/b2g/nightly/latest-mozilla-central/
> b2g-20.0a1.en-US.win32.zip.
Turns out it's not...we didn't get win32 localizer builds all weekend. Strangely, the part that fails appears to be doing the same thing as the dev build:
rm -f gkconhtmlcon_s.lib
e:/builds/moz2_slave/m-beta-w32-gecko-ntly/build/obj-firefox/_virtualenv/Scripts/python.exe /e/builds/moz2_slave/m-beta-w32-gecko-ntly/build/config/pythonpath.py -I../../../../config /e/builds/moz2_slave/m-beta-w32-gecko-ntly/build/config/expandlibs_gen.py --depend .deps/gkconhtmlcon_s.lib.desc.pp -o gkconhtmlcon_s.lib.desc HTMLPropertiesCollection.obj nsClientRect.obj nsHTMLDNSPrefetch.obj <snip>
make[8]: Leaving directory `/e/builds/moz2_slave/m-beta-w32-gecko-ntly/build/obj-firefox/content/html/content/src'
make -C test libs
make[8]: Entering directory `/e/builds/moz2_slave/m-beta-w32-gecko-ntly/build/obj-firefox/content/html/content/test'
e:/builds/moz2_slave/m-beta-w32-gecko-ntly/build/obj-firefox/_virtualenv/Scripts/python.exe /e/builds/moz2_slave/m-beta-w32-gecko-ntly/build/config/nsinstall.py -t <snip>
vs.
rm -f gkconhtmlcon_s.lib
e:/builds/moz2_slave/m-beta-w32-gecko-localizer-ntly/build/obj-firefox/_virtualenv/Scripts/python.exe /e/builds/moz2_slave/m-beta-w32-gecko-localizer-ntly/build/config/pythonpath.py -I../../../../config /e/builds/moz2_slave/m-beta-w32-gecko-localizer-ntly/build/config/expandlibs_gen.py --depend .deps/gkconhtmlcon_s.lib.desc.pp -o gkconhtmlcon_s.lib.desc HTMLPropertiesCollection.obj nsClientRect.obj <snip>
make[8]: Leaving directory `/e/builds/moz2_slave/m-beta-w32-gecko-localizer-ntly/build/obj-firefox/content/html/content/src'
make -C test libs
make[8]: Entering directory `/e/builds/moz2_slave/m-beta-w32-gecko-localizer-ntly/build/obj-firefox/content/html/content/test'
e:/builds/moz2_slave/m-beta-w32-gecko-localizer-ntly/build/obj-firefox/_virtualenv/Scripts/python.exe /e/builds/moz2_slave/m-beta-w32-gecko-localizer-ntly/build/config/nsinstall.py -t <snip>
make[8]: Leaving directory `/e/builds/moz2_slave/m-beta-w32-gecko-localizer-ntly/build/obj-firefox/content/html/content/test'
make[7]: Leaving directory `/e/builds/moz2_slave/m-beta-w32-gecko-localizer-ntly/build/obj-firefox/content/html/content'
make[6]: Leaving directory `/e/builds/moz2_slave/m-beta-w32-gecko-localizer-ntly/build/obj-firefox/content/html'
make[5]: Leaving directory `/e/builds/moz2_slave/m-beta-w32-gecko-localizer-ntly/build/obj-firefox/content'
make[4]: Leaving directory `/e/builds/moz2_slave/m-beta-w32-gecko-localizer-ntly/build/obj-firefox'
make[3]: Leaving directory `/e/builds/moz2_slave/m-beta-w32-gecko-localizer-ntly/build/obj-firefox'
make[2]: Leaving directory `/e/builds/moz2_slave/m-beta-w32-gecko-localizer-ntly/build/obj-firefox'
make[1]: Leaving directory `/e/builds/moz2_slave/m-beta-w32-gecko-localizer-ntly/build'
/bin/sh: e:/builds/moz2_slave/m-beta-w32-gecko-localizer-ntly/build/obj-firefox/_virtualenv/Scripts/python.exe: Bad file number
Comment 51•12 years ago
|
||
In pproduction
Comment 52•12 years ago
|
||
(In reply to Rail Aliiev [:rail] from comment #51)
> In pproduction
I production, of course, there is no such thing as pproduction. :)
Assignee | ||
Updated•12 years ago
|
Summary: Set up nightly multilocale B2G desktop builds for l10n testing → Set up nightly multilocale B2G desktop builds for dev+l10n testing
Assignee | ||
Comment 54•12 years ago
|
||
https://bugzilla.mozilla.org/show_bug.cgi?id=796051#c27 has some quick and dirty thoughts from Axel and how to integrate with the Gecko bits.
Assignee | ||
Updated•12 years ago
|
Assignee | ||
Comment 55•12 years ago
|
||
Attachment #688302 -
Flags: review?(aki)
Assignee | ||
Updated•12 years ago
|
Attachment #688302 -
Attachment is obsolete: true
Attachment #688302 -
Flags: review?(aki)
Assignee | ||
Comment 56•12 years ago
|
||
Attachment #688303 -
Flags: review?(aki)
Comment 57•12 years ago
|
||
Comment on attachment 688303 [details] [diff] [review]
be more liberal with win32 path finding
Tests, crashreporter zip will match. Ben's working on another patch.
Attachment #688303 -
Flags: review?(aki) → review-
Assignee | ||
Comment 58•12 years ago
|
||
Turns out we don't care about these properties for the MBF based b2g builds. May as well just not run this step. I hate this patch.
Attachment #688303 -
Attachment is obsolete: true
Attachment #688330 -
Flags: review?(aki)
Comment 59•12 years ago
|
||
Comment on attachment 688330 [details] [diff] [review]
hacky hacky hack hack for b2g
If it's any consolation, b2g desktop builds are a prime candidate to move to ScriptFactory in the not-distant-future.
Attachment #688330 -
Flags: review?(aki) → review+
Assignee | ||
Updated•12 years ago
|
Attachment #688330 -
Flags: checked-in+
Assignee | ||
Comment 60•12 years ago
|
||
Just want to summarize where we're at here, because there's so much churn:
* B2G Desktop builds for devs and localizers are both enabled on all platforms (linux32, linux64, mac64, win32)
* Bug 817170 is tracking an issue on all platforms that causes a black screen to show up instead of the FTU or home screen. Fabrice is looking into it. Until this is fixed these builds are effectively useless.
* Gecko is still en-US only. It's now in question whether or not that's going to be a blocker for B2G 1.0. More on that in bug 817917. If it ends up being a blocker there will be additional work here to enable it for b2g desktop builds.
Comment 61•12 years ago
|
||
(In reply to Ben Hearsum [:bhearsum] from comment #60)
> * Gecko is still en-US only. It's now in question whether or not that's
> going to be a blocker for B2G 1.0. More on that in bug 817917. If it ends up
> being a blocker there will be additional work here to enable it for b2g
> desktop builds.
I think you mean bug 817197.
Per Dietrich, he can't imagine that gecko multilocale wouldn't be a blocker, due to our target 1.0 audiences.
Assignee | ||
Comment 62•12 years ago
|
||
(In reply to Aki Sasaki [:aki] from comment #61)
> (In reply to Ben Hearsum [:bhearsum] from comment #60)
> > * Gecko is still en-US only. It's now in question whether or not that's
> > going to be a blocker for B2G 1.0. More on that in bug 817917. If it ends up
> > being a blocker there will be additional work here to enable it for b2g
> > desktop builds.
>
> I think you mean bug 817197.
>
> Per Dietrich, he can't imagine that gecko multilocale wouldn't be a blocker,
> due to our target 1.0 audiences.
OK, good to know. I got pulled into a triage meeting today and I was just echo'ing what was said there.
Updated•12 years ago
|
Comment 63•12 years ago
|
||
Comment 64•12 years ago
|
||
using http://hg.mozilla.org/users/asasaki_mozilla.com/mozharness
The test runs take a while, but I was when the macosx64 build died during the multilocale step, I was able to ssh in and
python mozharness/scripts/clone_gaia_locales.py --config-file multi_locale/b2g_macosx64.py --merge-locales --no-pull --build --locale es-ES
successfully.
--merge-locales is from the previous factory logic, so I added a dummy arg to allow that. --no-pull shouldn't even be necessary, I'm hitting a bug somewhere :( I'm specifying --locale es-ES because l10n-central/pt-BR is broken against the mobile strings, evidently :(
I made some more changes that *might* finish successfully, triggered another win32 and osx nightly on dev-master01.build.mozilla.org:8052
http://dev-master01.build.mozilla.org:8052/builders/b2g_mozilla-central_win32_gecko%20nightly
http://dev-master01.build.mozilla.org:8052/builders/b2g_mozilla-central_macosx64_gecko%20nightly
We probably want to rename the clone_gaia_locales.py script if we're going to use it this way.
This is far from polished here, but getting closer to something that'll produce a multilocale build in staging.
Ben -- hoping this is enough to get you started in your morning. I've got 10am and 12pm meetings Pacific, but otherwise will have time to talk w/ you about this.
Comment 65•12 years ago
|
||
Oh yeah, I had to use b2g/locales/all-locales because the gaia languages json has 'es' and b2g/locales/all-locales (and l10n-central) has 'es-ES', so I decided to just stick with the two separate locale files. :\
Comment 66•12 years ago
|
||
I may have fixed the pt-BR issue with http://hg.mozilla.org/users/asasaki_mozilla.com/mozharness/rev/6da6d97ae53e , but I'm not about to kill the current builds mid-build to verify. I triggered a new build for both osx and win32 and I'll look at all 4 builds in the morning.
Comment 67•12 years ago
|
||
To comment on http://hg.mozilla.org/users/asasaki_mozilla.com/mozharness/rev/6da6d97ae53e#l1.61, that's excluding en-US because it's en-US, and 'multi', because that's the locale code you pass in as AB_CD when doing the multi-locale repack.
Assignee | ||
Comment 68•12 years ago
|
||
Comment on attachment 689101 [details] [diff] [review]
wip buildbot-configs desktop gecko multilocale
Aki, sorry for not posting this yesterday but I got started on this just a little bit. I was looking at using the existing "enable_l10n" flag, but it implies a LOT of things in misc.py - including dep/nightly l10n builders. I don't think we gain a lot by using this flag. I was thinking that we should just pass the multilocale options plus the gecko l10n root, and that should be enough. I'm going to take a stab at that this morning - I'll bring your patches into mine too and then maybe we can sync up in a bit?
Assignee | ||
Comment 69•12 years ago
|
||
(In reply to Ben Hearsum [:bhearsum] from comment #68)
> Comment on attachment 689101 [details] [diff] [review]
> wip buildbot-configs desktop gecko multilocale
>
> Aki, sorry for not posting this yesterday but I got started on this just a
> little bit. I was looking at using the existing "enable_l10n" flag, but it
> implies a LOT of things in misc.py - including dep/nightly l10n builders. I
> don't think we gain a lot by using this flag. I was thinking that we should
> just pass the multilocale options plus the gecko l10n root, and that should
> be enough. I'm going to take a stab at that this morning - I'll bring your
> patches into mine too and then maybe we can sync up in a bit?
What I've got so far is here:
https://github.com/bhearsum/buildbotcustom/compare/master...gecko-multilocale
https://github.com/bhearsum/buildbot-configs/compare/master...gecko-multilocale
Haven't tested it yet.
Assignee | ||
Comment 70•12 years ago
|
||
Status update: Multilocale Gaia is now enabled for the RelEng B2G desktop builds (all branches).
Still left to do here is enabling multilocale Gecko after dependent bugs are fixed.
Assignee | ||
Comment 71•12 years ago
|
||
Been testing this on my dev master and it seems to be working fine. I need to do final tests on Monday and also validate the builds (!). Most of this patch was done by Aki, with small touchups by me, so I feel that we need a 3rd person to review, too.
Attachment #689906 -
Flags: review?(rail)
Attachment #689906 -
Flags: review?(aki)
Assignee | ||
Comment 72•12 years ago
|
||
Based off of Aki's patch. The changes I made to it, roughly:
* Pass geckoL10nRoot/geckoLanguagesFile
* Pass multiargs to dep+nightly factories. I realize now that this isn't strictly necessary because the only dep b2g builds we have are fully mozharness based, but there's wrong with doing it.
* Don't run file properties steps for b2g multilocale. HACK HACK HACK HACK HACK HACK HACK HACK HACK
Attachment #689101 -
Attachment is obsolete: true
Attachment #689102 -
Attachment is obsolete: true
Attachment #689913 -
Flags: review?(rail)
Attachment #689913 -
Flags: review?(aki)
Assignee | ||
Comment 73•12 years ago
|
||
Not a whole lot of change since Aki's original patch:
* Pass and set gecko languages file. We need it in the multi options because that's what's used when we're *building* the multilocale build. We also need it in the platform because that's how geckoL10nRoot gets set, and that's used when *cloning* the repos. Might be better if we only clone gaia before the build, and let the multilocale script do the gecko cloning? If we did that we could rid of the duplication.
* Add TODOs for device builds.
* Set gecko_l10n_root. I think i still need to do this for try.
Attachment #689914 -
Flags: review?(rail)
Attachment #689914 -
Flags: review?(aki)
Comment 74•12 years ago
|
||
Comment on attachment 689906 [details] [diff] [review]
mozharness changes for gecko multilocale
>diff --git a/scripts/b2g_multilocale.py b/scripts/b2g_multilocale.py
>new file mode 100755
>index 0000000..f077572
This might be a confusing name since it's only used for desktop multilocale. Maybe a comment at the top if we don't rename?
>--- /dev/null
>+++ b/scripts/b2g_multilocale.py
>@@ -0,0 +1,192 @@
>+#!/usr/bin/env python
>+# Mozilla licence shtuff
Hah, I missed this in clone_gaia_locales.py. Probably ok, maybe we should paste it in?
>diff --git a/scripts/clone_gaia_locales.py b/scripts/clone_gaia_locales.py
>deleted file mode 100644
>index 0ee2c77..0000000
>--- a/scripts/clone_gaia_locales.py
>+++ /dev/null
We may have a timing issue with landing this and the reconfig.
I recommend landing in 3 phases:
1. Land mozharness bits without removing this file
2. Land buildbot bits + reconfig
3. Remove this file
to avoid burning.
Attachment #689906 -
Flags: review?(aki) → review+
Comment 75•12 years ago
|
||
Then again, since these builds are nightly-only, as long as all the patches land + we reconfig before the nightly, that's fine too.
Comment 76•12 years ago
|
||
Comment on attachment 689914 [details] [diff] [review]
buildbot-configs for gecko multilocale
I think the thing I'm most worried about here is enable_multi_locale and enable_l10n is specified at the branch level in mozilla/config.py, which lets us specify certain branches as l10n-enabled and multilocale-enabled. (e.g., inbound and project branches aren't).
This + the buildbotcustom script move towards only looking at the enable_multi_locale flag, which is probably fine, since I can't think of a branch that would have multilocale without l10n at all.
... now that I'm thinking more about it it's probably fine.
Attachment #689914 -
Flags: review?(aki) → review+
Comment 77•12 years ago
|
||
Comment on attachment 689913 [details] [diff] [review]
buildbotcustom bits for gecko multilocale
Ok, now I'm wondering if multiargs in dep builds is going to have a negative effect on Android builds, which are multilocale for nightlies only.
enable_multilocale_dep ?
if pf.get('product_name') == 'b2g': ?
Attachment #689913 -
Flags: review?(aki) → review+
Assignee | ||
Comment 78•12 years ago
|
||
(In reply to Aki Sasaki [:aki] from comment #77)
> Comment on attachment 689913 [details] [diff] [review]
> buildbotcustom bits for gecko multilocale
>
> Ok, now I'm wondering if multiargs in dep builds is going to have a negative
> effect on Android builds, which are multilocale for nightlies only.
>
> enable_multilocale_dep ?
> if pf.get('product_name') == 'b2g': ?
To be honest, I can just not pass multiargs to the dep builds. I put that part in thinking we would need it for device builds. But since those are ScriptFactory based we obviously don't.
Comment 79•12 years ago
|
||
Comment on attachment 689906 [details] [diff] [review]
mozharness changes for gecko multilocale
Review of attachment 689906 [details] [diff] [review]:
-----------------------------------------------------------------
Just a couple of nits.
::: scripts/b2g_multilocale.py
@@ +1,2 @@
> +#!/usr/bin/env python
> +# Mozilla licence shtuff
Replace with real Mozilla licence shtuff?
@@ +19,5 @@
> + """ This is a helper script that requires MercurialBuildFactory
> + logic to work. We may eventually make this a standalone
> + script.
> +
> + TODO: rename
No sure if you still need this TODO...
@@ +23,5 @@
> + TODO: rename
> + We could inherit MercurialScript instead of BaseScript + VCSMixin
> + """
> + config_options = [
> + [["--locale",], {
"--locale", - you can drop that extra coma
Attachment #689906 -
Flags: review?(rail) → review+
Updated•12 years ago
|
Attachment #689913 -
Flags: review?(rail) → review+
Comment 80•12 years ago
|
||
Comment on attachment 689914 [details] [diff] [review]
buildbot-configs for gecko multilocale
Review of attachment 689914 [details] [diff] [review]:
-----------------------------------------------------------------
::: mozilla/b2g_config.py
@@ +222,5 @@
> ],
> + 'multi_locale': True,
> + 'multi_config_name': 'multi_locale/b2g_linux32.py',
> + 'mozharness_multi_options': [
> + '--no-pull', # TODO figure out why it isn't working w/out this
Can you test if you really need to disable pull here?
Attachment #689914 -
Flags: review?(rail) → review+
Assignee | ||
Comment 81•12 years ago
|
||
(In reply to Aki Sasaki [:aki] from comment #74)
> Comment on attachment 689906 [details] [diff] [review]
> mozharness changes for gecko multilocale
>
> >diff --git a/scripts/b2g_multilocale.py b/scripts/b2g_multilocale.py
> >new file mode 100755
> >index 0000000..f077572
>
> This might be a confusing name since it's only used for desktop multilocale.
> Maybe a comment at the top if we don't rename?
I'll just rename it.
(In reply to Aki Sasaki [:aki] from comment #75)
> Then again, since these builds are nightly-only, as long as all the patches
> land + we reconfig before the nightly, that's fine too.
Yeah, I think we'll be okay. I plan to reconfig immediately for these changes.
(In reply to Aki Sasaki [:aki] from comment #76)
> Comment on attachment 689914 [details] [diff] [review]
> buildbot-configs for gecko multilocale
>
> I think the thing I'm most worried about here is enable_multi_locale and
> enable_l10n is specified at the branch level in mozilla/config.py, which
> lets us specify certain branches as l10n-enabled and multilocale-enabled.
> (e.g., inbound and project branches aren't).
Here's the thing though, "enable_l10n" is really just a poorly named "enable desktop l10n" or "enable single locale l10n" when you look at what it actually does. Personally, I see multilocale as nearly a completely separate thing than l10n.
And for B2G, multilocale isn't really even an option. All builds should be multilocale. Even dep device builds will be multilocale. There's no point in doing any other kind.
> This + the buildbotcustom script move towards only looking at the
> enable_multi_locale flag, which is probably fine, since I can't think of a
> branch that would have multilocale without l10n at all.
>
> ... now that I'm thinking more about it it's probably fine.
OK. So to summarize: I don't need to make any additional changes based on this feedback?
Assignee | ||
Comment 82•12 years ago
|
||
I've got review comments addressed locally and I'm doing some final tests. If they succeed we should be good to land today.
Assignee | ||
Comment 83•12 years ago
|
||
Renamed the script
Added license block
Removed superfluous comma
Attachment #689906 -
Attachment is obsolete: true
Attachment #690453 -
Flags: review?(rail)
Assignee | ||
Comment 84•12 years ago
|
||
Attachment #690454 -
Flags: review?(aki)
Assignee | ||
Comment 85•12 years ago
|
||
Attachment #689913 -
Attachment is obsolete: true
Attachment #689914 -
Attachment is obsolete: true
Attachment #690455 -
Flags: review?(aki)
Comment 86•12 years ago
|
||
Comment on attachment 690453 [details] [diff] [review]
rename script, fix nits
lgtm
Attachment #690453 -
Flags: review?(rail) → review+
Comment 87•12 years ago
|
||
Comment on attachment 690455 [details] [diff] [review]
remove --no-pull
Hunh, that was bothering me but I didn't investigate further.
Glad that it went away.
Attachment #690455 -
Flags: review?(aki) → review+
Updated•12 years ago
|
Attachment #690454 -
Flags: review?(aki) → review+
Assignee | ||
Comment 88•12 years ago
|
||
Comment on attachment 690455 [details] [diff] [review]
remove --no-pull
I noticed that b2g18 landed since I made this patch, I added gecko_l10n_root for it, too when I landed this.
Attachment #690455 -
Flags: checked-in+
Assignee | ||
Updated•12 years ago
|
Attachment #690454 -
Flags: checked-in+
Assignee | ||
Comment 89•12 years ago
|
||
Comment on attachment 690453 [details] [diff] [review]
rename script, fix nits
Patches are landed and in production, started some new desktop nightlies.
Attachment #690453 -
Flags: checked-in+
Assignee | ||
Comment 90•12 years ago
|
||
Waiting on bugs 796051, 808326, and 817197 to be backported. Axel wants to test on central first.
Attachment #690551 -
Flags: review?(aki)
Assignee | ||
Updated•12 years ago
|
Updated•12 years ago
|
Attachment #690551 -
Flags: review?(aki) → review+
Assignee | ||
Comment 91•12 years ago
|
||
We need to flip enable_multi_locale too, otherwise we'll still try to do the builds, because the mozharness script has a default set for locales_file.
Attachment #690557 -
Flags: review?(aki)
Updated•12 years ago
|
Attachment #690557 -
Flags: review?(aki) → review+
Assignee | ||
Updated•12 years ago
|
Attachment #690557 -
Flags: checked-in+
Assignee | ||
Comment 92•12 years ago
|
||
Attachment #690576 -
Flags: review?(aki)
Comment 93•12 years ago
|
||
Comment on attachment 690576 [details] [diff] [review]
super disable
super disable!
Attachment #690576 -
Flags: review?(aki) → review+
Assignee | ||
Updated•12 years ago
|
Attachment #690576 -
Flags: checked-in+
Comment 94•12 years ago
|
||
Looking at bug 820160, it'd be great to get a new build triggered to test. We might have just picked a bad hour for our desktop build.
Central, aka 20, as previously.
Comment 95•12 years ago
|
||
Triggered a set of m-c desktop b2g nightlies (developer, not localizer, i.e. 6 gaia locales).
Assignee | ||
Comment 96•12 years ago
|
||
Hope to have this bug closed out by EOD. The only thing left to do here is enable Gecko multilocale on aurora, beta, and b2g18. We're waiting for the Gecko bits to be uplifted first, which are in bugs 796051 and 817197.
Assignee | ||
Comment 97•12 years ago
|
||
Meant to do this in the last patch I landed, but missed it.
Attachment #691857 -
Flags: review?(aki)
Updated•12 years ago
|
Attachment #691857 -
Flags: review?(aki) → review+
Assignee | ||
Comment 98•12 years ago
|
||
Comment on attachment 691857 [details] [diff] [review]
re-enable multilocale for desktop builds
Landed, reconfiging for it now.
Attachment #691857 -
Flags: checked-in+
Assignee | ||
Comment 99•12 years ago
|
||
Okay, the last patch here is in production. Multilocale gecko and gaia is enabled for all b2g desktop builds now.
Status: NEW → RESOLVED
Closed: 12 years ago
Resolution: --- → FIXED
Updated•12 years ago
|
status-b2g18:
--- → fixed
Updated•11 years ago
|
Product: mozilla.org → Release Engineering
Updated•7 years ago
|
Component: General Automation → General
You need to log in
before you can comment on or make changes to this bug.
Description
•