Closed Bug 1728472 Opened 3 years ago Closed 3 years ago

Advertise supported languages in MSIX packages

Categories

(Firefox :: Installer, enhancement, P3)

enhancement

Tracking

()

RESOLVED FIXED
94 Branch
Tracking Status
firefox94 --- fixed

People

(Reporter: nalexander, Assigned: nalexander)

References

(Blocks 1 open bug)

Details

Attachments

(2 files)

Right now, the MSIX packages we produce advertise only support for the en-US locale. But the MSIX has lots of langpacks installed within it, so it should advertise support for more locale codes. This ticket tracks updating the msix.py tooling to list the supported locales.

There's a wrinkle here that I didn't anticipate. My patch advertises support for a large number of locales: 99 for Firefox Release and Beta. But installing a try build fails because Windows doesn't recognize "ach" as a valid locale/language. Taking the list from https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-lcid/a9eac961-e77d-41a6-90a5-ce1a8b0cdb9c and intersecting our release locale list from https://hg.mozilla.org/releases/mozilla-release/file/552e8d1de16d462dfe0f3ab8688086d5072eaa92/browser/locales/shipped-locales, I find the following locale codes are not present:

ach
an
ca-valencia
cak
ja-JP-mac
lij
sco
son
szl
tl
trs

Maybe ja-JP-mac will work since ja-JP is present; and maybe ca-valencia will work because ca is present; but the other ones probably not.

How would l10n-team like me to handle this? I can add an allow-list with Microsoft's published data, or a deny-list with the items listed here, or something else? I'd prefer the deny-list since it's smaller.

Flags: needinfo?(francesco.lodolo)

The list looks similar to the list of release locales that are not supported by CLDR (minus gn and oc, which is surprising):

ach
an
cak
gn
lij
oc
sco
son
szl
tl
trs

ja-JP-mac should be excluded for sure, since it's a langpack that we only use for macOS, and Japanese is covered by ja. We should also make sure to not package it.

What I don't fully understand is what a deny-list implies: is the build still going to package the langpack, but not publishing that information as metadata?

A deny-list is shorter, but it's also likely that new locales are going to be in this list (because we mostly add minority languages at this point).

Is there a way to programmatically access the list of supported locales by Microsoft? Both a deny-list  and an  accept-list can suddenly include obsolete items, and we'll never know. If that's not possible, a deny-list seems like the easiest solution.

Flags: needinfo?(francesco.lodolo)

ja-JP-mac should be excluded for sure, since it's a langpack that we only use for macOS, and Japanese is covered by ja. We should also make sure to not package it.

I can arrange this.

What I don't fully understand is what a deny-list implies: is the build still going to package the langpack, but not publishing that information as metadata?

We have a choice here. Given our earlier discussions about how manually installed langpacks would interact with MSIX updates, which are paveovers, it seems like we should package all the langpacks (without ja-JP-mac) and just advertise a subset.

A deny-list is shorter, but it's also likely that new locales are going to be in this list (because we mostly add minority languages at this point).

Is there a way to programmatically access the list of supported locales by Microsoft? Both a deny-list  and an  accept-list can suddenly include obsolete items, and we'll never know. If that's not possible, a deny-list seems like the easiest solution.

I'm not aware of a way to access this list dynamically. The set of locales supported grows slowly, and since we need to target Windows 10+, it's just dictated by the set of locales supported by the initial release of Windows 10.

I think we can roll with a deny-list for now and leave improvements to the future, should it arrive. Thanks, :flod!

Priority: -- → P3
Attachment #9239038 - Attachment description: Bug 1728472 - Advertise supported languages in MSIX packages. r?bhearsum! → Bug 1728472 - Part 1: Advertise supported languages in MSIX packages. r?bhearsum!

Windows MSIX packages support a finite set of locales: see
https://docs.microsoft.com/en-us/windows/uwp/publish/supported-languages.
This patch encodes that list in
browser/installer/windows/msix/msix-all-locales. Two ad-hoc
modifications were necessary: removing 'sr*' (Serbian) and 'uz*'
(Uzbek) in order for the resulting MSIX packages to install.

We distribute all of the langpacks supported by the release channel in
our MSIX, which is encoded in browser/locales/all-locales. But we
only advertise support in the App manifest for the intersection of
that set and the set of locales supported by Windows.

We do so to avoid the following issue. Suppose a user manually
installs a langpack that is not supported by Windows, and then updates
the installed MSIX package. MSIX package upgrades are essentially
paveover installs, so there is no opportunity for Firefox to update
the langpack before the update. But, since all langpacks are bundled
with the MSIX, that langpack will be up-to-date, preventing one class
of YSOD.

Depends on D124263

For the record, when I repack a recent Beta, the distributed languages and the ones not advertised are:

$ WINDOWSSDKDIR='C:/Program Files (x86)/Windows Kits/10' ./mach repackage msix --input ../Beta/VFS/ProgramFiles/Firefox\ Beta\ Package\ Root --channel=beta --arch=x86_64 --sign --verbose --distribution ../Beta/distribution
application.ini version is '93.0' and build ID is '20210920185952': embedded version will be '93.0.0.0'
Repackaging to: c:/Users/nalexander/.mozbuild/cache/mach-msix/Mozilla.MozillaFirefoxBeta_93.0.0.0_x64.msix
Adding distribution files from c:/Users/nalexander/Mozilla/gecko/browser/installer/windows/msix/distribution
Adding distribution files from ../Beta/distribution
Distributing locale 'ach' from extensions/langpack-ach@firefox.mozilla.org.xpi
Distributing locale 'af' from extensions/langpack-af@firefox.mozilla.org.xpi
Distributing locale 'an' from extensions/langpack-an@firefox.mozilla.org.xpi
Distributing locale 'ar' from extensions/langpack-ar@firefox.mozilla.org.xpi
Distributing locale 'ast' from extensions/langpack-ast@firefox.mozilla.org.xpi
Distributing locale 'az' from extensions/langpack-az@firefox.mozilla.org.xpi
Distributing locale 'be' from extensions/langpack-be@firefox.mozilla.org.xpi
Distributing locale 'bg' from extensions/langpack-bg@firefox.mozilla.org.xpi
Distributing locale 'bn' from extensions/langpack-bn@firefox.mozilla.org.xpi
Distributing locale 'br' from extensions/langpack-br@firefox.mozilla.org.xpi
Distributing locale 'bs' from extensions/langpack-bs@firefox.mozilla.org.xpi
Distributing locale 'ca-valencia' from extensions/langpack-ca-valencia@firefox.mozilla.org.xpi
Distributing locale 'ca' from extensions/langpack-ca@firefox.mozilla.org.xpi
Distributing locale 'cak' from extensions/langpack-cak@firefox.mozilla.org.xpi
Distributing locale 'cs' from extensions/langpack-cs@firefox.mozilla.org.xpi
Distributing locale 'cy' from extensions/langpack-cy@firefox.mozilla.org.xpi
Distributing locale 'da' from extensions/langpack-da@firefox.mozilla.org.xpi
Distributing locale 'de' from extensions/langpack-de@firefox.mozilla.org.xpi
Distributing locale 'dsb' from extensions/langpack-dsb@firefox.mozilla.org.xpi
Distributing locale 'el' from extensions/langpack-el@firefox.mozilla.org.xpi
Distributing locale 'en-ca' from extensions/langpack-en-CA@firefox.mozilla.org.xpi
Distributing locale 'en-gb' from extensions/langpack-en-GB@firefox.mozilla.org.xpi
Distributing locale 'eo' from extensions/langpack-eo@firefox.mozilla.org.xpi
Distributing locale 'es-ar' from extensions/langpack-es-AR@firefox.mozilla.org.xpi
Distributing locale 'es-cl' from extensions/langpack-es-CL@firefox.mozilla.org.xpi
Distributing locale 'es-es' from extensions/langpack-es-ES@firefox.mozilla.org.xpi
Distributing locale 'es-mx' from extensions/langpack-es-MX@firefox.mozilla.org.xpi
Distributing locale 'et' from extensions/langpack-et@firefox.mozilla.org.xpi
Distributing locale 'eu' from extensions/langpack-eu@firefox.mozilla.org.xpi
Distributing locale 'fa' from extensions/langpack-fa@firefox.mozilla.org.xpi
Distributing locale 'ff' from extensions/langpack-ff@firefox.mozilla.org.xpi
Distributing locale 'fi' from extensions/langpack-fi@firefox.mozilla.org.xpi
Distributing locale 'fr' from extensions/langpack-fr@firefox.mozilla.org.xpi
Distributing locale 'fy-nl' from extensions/langpack-fy-NL@firefox.mozilla.org.xpi
Distributing locale 'ga-ie' from extensions/langpack-ga-IE@firefox.mozilla.org.xpi
Distributing locale 'gd' from extensions/langpack-gd@firefox.mozilla.org.xpi
Distributing locale 'gl' from extensions/langpack-gl@firefox.mozilla.org.xpi
Distributing locale 'gn' from extensions/langpack-gn@firefox.mozilla.org.xpi
Distributing locale 'gu-in' from extensions/langpack-gu-IN@firefox.mozilla.org.xpi
Distributing locale 'he' from extensions/langpack-he@firefox.mozilla.org.xpi
Distributing locale 'hi-in' from extensions/langpack-hi-IN@firefox.mozilla.org.xpi
Distributing locale 'hr' from extensions/langpack-hr@firefox.mozilla.org.xpi
Distributing locale 'hsb' from extensions/langpack-hsb@firefox.mozilla.org.xpi
Distributing locale 'hu' from extensions/langpack-hu@firefox.mozilla.org.xpi
Distributing locale 'hy-am' from extensions/langpack-hy-AM@firefox.mozilla.org.xpi
Distributing locale 'ia' from extensions/langpack-ia@firefox.mozilla.org.xpi
Distributing locale 'id' from extensions/langpack-id@firefox.mozilla.org.xpi
Distributing locale 'is' from extensions/langpack-is@firefox.mozilla.org.xpi
Distributing locale 'it' from extensions/langpack-it@firefox.mozilla.org.xpi
Distributing locale 'ja' from extensions/langpack-ja@firefox.mozilla.org.xpi
Distributing locale 'ka' from extensions/langpack-ka@firefox.mozilla.org.xpi
Distributing locale 'kab' from extensions/langpack-kab@firefox.mozilla.org.xpi
Distributing locale 'kk' from extensions/langpack-kk@firefox.mozilla.org.xpi
Distributing locale 'km' from extensions/langpack-km@firefox.mozilla.org.xpi
Distributing locale 'kn' from extensions/langpack-kn@firefox.mozilla.org.xpi
Distributing locale 'ko' from extensions/langpack-ko@firefox.mozilla.org.xpi
Distributing locale 'lij' from extensions/langpack-lij@firefox.mozilla.org.xpi
Distributing locale 'lt' from extensions/langpack-lt@firefox.mozilla.org.xpi
Distributing locale 'lv' from extensions/langpack-lv@firefox.mozilla.org.xpi
Distributing locale 'mk' from extensions/langpack-mk@firefox.mozilla.org.xpi
Distributing locale 'mr' from extensions/langpack-mr@firefox.mozilla.org.xpi
Distributing locale 'ms' from extensions/langpack-ms@firefox.mozilla.org.xpi
Distributing locale 'my' from extensions/langpack-my@firefox.mozilla.org.xpi
Distributing locale 'nb-no' from extensions/langpack-nb-NO@firefox.mozilla.org.xpi
Distributing locale 'ne-np' from extensions/langpack-ne-NP@firefox.mozilla.org.xpi
Distributing locale 'nl' from extensions/langpack-nl@firefox.mozilla.org.xpi
Distributing locale 'nn-no' from extensions/langpack-nn-NO@firefox.mozilla.org.xpi
Distributing locale 'oc' from extensions/langpack-oc@firefox.mozilla.org.xpi
Distributing locale 'pa-in' from extensions/langpack-pa-IN@firefox.mozilla.org.xpi
Distributing locale 'pl' from extensions/langpack-pl@firefox.mozilla.org.xpi
Distributing locale 'pt-br' from extensions/langpack-pt-BR@firefox.mozilla.org.xpi
Distributing locale 'pt-pt' from extensions/langpack-pt-PT@firefox.mozilla.org.xpi
Distributing locale 'rm' from extensions/langpack-rm@firefox.mozilla.org.xpi
Distributing locale 'ro' from extensions/langpack-ro@firefox.mozilla.org.xpi
Distributing locale 'ru' from extensions/langpack-ru@firefox.mozilla.org.xpi
Distributing locale 'sco' from extensions/langpack-sco@firefox.mozilla.org.xpi
Distributing locale 'si' from extensions/langpack-si@firefox.mozilla.org.xpi
Distributing locale 'sk' from extensions/langpack-sk@firefox.mozilla.org.xpi
Distributing locale 'sl' from extensions/langpack-sl@firefox.mozilla.org.xpi
Distributing locale 'son' from extensions/langpack-son@firefox.mozilla.org.xpi
Distributing locale 'sq' from extensions/langpack-sq@firefox.mozilla.org.xpi
Distributing locale 'sr' from extensions/langpack-sr@firefox.mozilla.org.xpi
Distributing locale 'sv-se' from extensions/langpack-sv-SE@firefox.mozilla.org.xpi
Distributing locale 'szl' from extensions/langpack-szl@firefox.mozilla.org.xpi
Distributing locale 'ta' from extensions/langpack-ta@firefox.mozilla.org.xpi
Distributing locale 'te' from extensions/langpack-te@firefox.mozilla.org.xpi
Distributing locale 'th' from extensions/langpack-th@firefox.mozilla.org.xpi
Distributing locale 'tl' from extensions/langpack-tl@firefox.mozilla.org.xpi
Distributing locale 'tr' from extensions/langpack-tr@firefox.mozilla.org.xpi
Distributing locale 'trs' from extensions/langpack-trs@firefox.mozilla.org.xpi
Distributing locale 'uk' from extensions/langpack-uk@firefox.mozilla.org.xpi
Distributing locale 'ur' from extensions/langpack-ur@firefox.mozilla.org.xpi
Distributing locale 'uz' from extensions/langpack-uz@firefox.mozilla.org.xpi
Distributing locale 'vi' from extensions/langpack-vi@firefox.mozilla.org.xpi
Distributing locale 'xh' from extensions/langpack-xh@firefox.mozilla.org.xpi
Distributing locale 'zh-cn' from extensions/langpack-zh-CN@firefox.mozilla.org.xpi
Distributing locale 'zh-tw' from extensions/langpack-zh-TW@firefox.mozilla.org.xpi
Not advertising distributed locale 'ach' that is not recognized by Windows
Not advertising distributed locale 'an' that is not recognized by Windows
Not advertising distributed locale 'ast' that is not recognized by Windows
Not advertising distributed locale 'az' that is not recognized by Windows
Not advertising distributed locale 'br' that is not recognized by Windows
Not advertising distributed locale 'ca-valencia' that is not recognized by Windows
Not advertising distributed locale 'cak' that is not recognized by Windows
Not advertising distributed locale 'dsb' that is not recognized by Windows
Not advertising distributed locale 'eo' that is not recognized by Windows
Not advertising distributed locale 'ff' that is not recognized by Windows
Not advertising distributed locale 'fy-nl' that is not recognized by Windows
Not advertising distributed locale 'gd' that is not recognized by Windows
Not advertising distributed locale 'gn' that is not recognized by Windows
Not advertising distributed locale 'hsb' that is not recognized by Windows
Not advertising distributed locale 'ia' that is not recognized by Windows
Not advertising distributed locale 'kab' that is not recognized by Windows
Not advertising distributed locale 'lij' that is not recognized by Windows
Not advertising distributed locale 'my' that is not recognized by Windows
Not advertising distributed locale 'oc' that is not recognized by Windows
Not advertising distributed locale 'rm' that is not recognized by Windows
Not advertising distributed locale 'sco' that is not recognized by Windows
Not advertising distributed locale 'son' that is not recognized by Windows
Not advertising distributed locale 'sr' that is not recognized by Windows
Not advertising distributed locale 'szl' that is not recognized by Windows
Not advertising distributed locale 'tl' that is not recognized by Windows
Not advertising distributed locale 'trs' that is not recognized by Windows
Not advertising distributed locale 'uz' that is not recognized by Windows
...
Pushed by nalexander@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/4b7390c0d217 Part 1: Advertise supported languages in MSIX packages. r=bhearsum https://hg.mozilla.org/integration/autoland/rev/a4035069a2e0 Part 2: Restrict advertised languages in MSIX to allow-list from Windows. r=bhearsum,flod
Status: ASSIGNED → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
Target Milestone: --- → 94 Branch
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: