Staś Małolepszy :stas Assignee
	Description • 10 years ago

Bug 1011519 tracks the work required to move all tests to use pseudolocales and disable ar, fr, and zh-TW in languages.json.  Once this work is done, it will be possible to remove all unused *.properties files.

	Kevin Grandon :kgrandon
	Comment 1 • 10 years ago

Seems it can still be useful to have *real* locale files in the codebase to test with. Sometimes bugs do not reproduce with psuedolocales, so it might be useful to have a second *real* locale in the codebase.

	Zibi Braniecki [:zbraniecki][:gandalf]
	Comment 2 • 10 years ago

I don't think that keeping a real locale in source makes any sense. On the other hand, I believe that generatic a pseudo-locale at build time adds value and covers cases which are not covered by runtime pseudolocale.

	Kevin Grandon :kgrandon
	Comment 3 • 10 years ago

If we move to build-time pseudo-locales (which I think we should probably do), then my concerns would be addressed, and I would support removing other locale folders. The biggest problem right now is that they behave differently, so sometimes developers need to go into a different language property file to hack it and test.

	Staś Małolepszy :stas Assignee
	Comment 4 • 10 years ago

Even if we generate pseudolocales on buildtime (which would be good for performance, but a bit contradicting the fact that they're a runtime setting right now), the files would be generated automatically right before they're put into application.zip, so it would be difficult to change something in them and test. To address the specific use case Kevin mentioned in comment 3, I think we'd need to introduce a new make target, say `make pseudolocalizations` which would generate the files before build time.  Note that right now the qps logic only operates on ASTs, while `make pseudolocalizations` would need to also serialize those ASTs to .properties.

An alternative solution might come from an admittedly unexpected angle.  I hear we might want to keep all localization in the Gaia git repo after all.  I'm not sure how developed this idea is, but if we were to keep the 'master' versions of each localization in Gaia repo then the synchronization issue would be be a problem, mostly.  Pseudolocalizations could still be used for RTL testing and when a new feature is developed for which localization don't exist yet.  Pike, what are your thoughts on this?

	Axel Hecht [:Pike]
	Comment 5 • 10 years ago

I'd run on the assumption that the l10n files stay outside of the gaia repo for the time being. Ad-hoc, in a post-Aisle world, we could consider that, but even then there's a bunch of technical and political stuff to clear out. Like trying to contract LSPs to commit to our github repo sounds challenging. Tree rules on frozen branches, etc etc etc.

	Staś Małolepszy :stas Assignee
	Comment 6 • 10 years ago

Thanks, Axel.  Looks like we won't be able to do what I had in mind in comment 4.

Going back to Kevin's comment 1:  I'm still not sure I like the idea of having even one additional language in the repo.  It would need to be synchronized periodically or would fall out of sync quickly and become a source of additional confusion when things don't work as expected.

I wonder if we could instead have a special make flag, INCLUDE_TEST_LOCALES, which when set would create a new temp dir, pull from https://hg.mozilla.org/gaia-l10n/fr and set LOCALES_FILE and LOCALE_BASEDIR such that the resulting build would include the freshly fetched localization files and source en-US.  Kevin, do you think this would be a good idea?

One challenge of this approach is how to make sure the files are fetched from the correct repo.  It's https://hg.mozilla.org/gaia-l10n for Gaia master, but https://hg.mozilla.org/releases/gaia-l10n/v2_1 for v2.1 etc.  I think we can start with master for now and figure this out later when we're getting ready to branch.

	Kevin Grandon :kgrandon
	Comment 7 • 10 years ago

My concerns are actually addressed if we move the pseudo-locales to use a build-time thing. The reason being is that generally we just need to "make sure things work", and today pseudo locales almost provide that, but the fall short. The last problem being when we ran into the issue with private browsing, where we weren't referencing the manifest.webapp. This worked fine for pseudo-locales, but did not work when applied to a real locale. Yes - there are ways we could patch this, and provide warnings to the user, but I feel they will always fall short of working like "real locales" in some way.

Basically if we have one additional real locale (or build-time pseudo-locales) then I'm happy. Using a build flag to fetch external locales might also work, but it seems overly complex to me.

	Staś Małolepszy :stas Assignee
	Comment 8 • 10 years ago

Thanks, Kevin.  This makes sense.  I've adjusted the strategy as follows:

 1. First, let's migrate all tests to use runtime pseudolocales (bug 1011519).
 2. Then, let's make it possible to build pseudolocales on build-time (bug 1101632).
 3. Finally, let's disable and remove ar, fr and zh-TW from the source repo (this bug).

	Staś Małolepszy :stas Assignee
	Comment 9 • 10 years ago

Bug 1101632 disabled ar, fr and zh-TW in dev builds.  Let's remove those outdated *.properties files.

	Staś Małolepszy :stas Assignee
	Comment 11 • 10 years ago

Comment on attachment 8568491 [details]
[gaia] stasm:1099321-remove-ar-fr-zh-TW > mozilla-b2g:master

Bye bye, *.{ar,fr,zh-TW}.properties :)

	Zibi Braniecki [:zbraniecki][:gandalf]
	Comment 12 • 10 years ago

Comment on attachment 8568491 [details]
[gaia] stasm:1099321-remove-ar-fr-zh-TW > mozilla-b2g:master

r++

	Staś Małolepszy :stas Assignee
	Comment 13 • 10 years ago

Comment on attachment 8568491 [details]
[gaia] stasm:1099321-remove-ar-fr-zh-TW > mozilla-b2g:master

Kevin, I'd like to run this by you as well.  The change only removes *.{ar,fr,zh-TW}.properties files in the repo.  I fixed the tests earlier in bug 1011519.

Treeherder: https://treeherder.mozilla.org/#/jobs?repo=gaia-try&revision=93c4078a670b

	Kevin Grandon :kgrandon
	Comment 14 • 10 years ago

Comment on attachment 8568491 [details]
[gaia] stasm:1099321-remove-ar-fr-zh-TW > mozilla-b2g:master

Nice work, looks good to me. Please make sure the try run looks ok before landing.

	Staś Małolepszy :stas Assignee
	Comment 15 • 10 years ago

Thanks.  I rebased one more time because I was getting an orange Gij.  This one's all green:

https://treeherder.mozilla.org/#/jobs?repo=gaia-try&revision=765c3bbcc5ba

	Staś Małolepszy :stas Assignee
	Comment 16 • 10 years ago

https://github.com/mozilla-b2g/gaia/commit/1cb43e424af904ce45bcf2bd312dd2c1ca03af2c

¯\_(ツ)_/¯_/¯(ツ)¯\_ ¯\_(ツ)¯\_ _/¯(ツ)_/¯

	Tim Guan-tin Chien [:timdream] (please needinfo)
	Comment 17 • 10 years ago

:stas,

I have noticed that HTML files still contains the following <meta> tag:

  <meta name="availableLanguages" content="ar, en-US, fr, zh-TW">

should they be removed too?

	Staś Małolepszy :stas Assignee
	Comment 18 • 10 years ago

You're right, Tim, thanks for pointing that out.  I filed bug 1139293.