Closed Bug 1738422 Opened 3 years ago Closed 3 years ago

Update our in-tree ICU to 70

Categories

(Core :: Internationalization, enhancement)

enhancement

Tracking

()

RESOLVED FIXED
96 Branch
Tracking Status
firefox95 --- wontfix
firefox96 --- fixed

People

(Reporter: anba, Assigned: anba)

References

(Regressed 1 open bug)

Details

Attachments

(16 files)

(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details

Also updates to CLDR 40 and Unicode 14 and brings various bug fixes and improvements.

Release notes: https://icu.unicode.org/download/70

Remove the patches for bug 1636984, bug 1648137, and bug 1714933 because the
underlying bugs were fixed in ICU 70 resp. the local patches were backports from
the ICU development branch.

And update the local patches for Sorbian to apply cleanly on ICU 70.

Update to ICU 70.1 by running "update-icu.sh" with "maint/maint-70" as the target.

Update to Unicode 14 by running ./make_unicode.py --version=UNIDATA.

Update to Unicode 14 by running genUnicodePropertyData.pl and genSpecialCasingData.pl.

Depends on D129925

Update language tag mappings by running ./make_intl_data.py langtags core.zip.

The direct invocation ./make_intl_data.py langtags doesn't work, because
"https://unicode.org/Public/cldr/40/core.zip" was incorrectly generated and
contains another "core.zip" file within "core.zip".

Depends on D129926

Add the new numbering system "tnsa" and then update numbering systems by
by running ./make_intl_data.py numbering.

Adding "tnsa" to ECMA-402 is tracked at https://github.com/tc39/ecma402/pull/614.
ECMA-402 allows to support additional numbering systems, so even before that PR,
it's okay to support "tnsa". (https://tc39.es/ecma402/#annex-implementation-dependent-behaviour)

Depends on D129927

Update measurement units by running ./make_intl_data.py units.

Depends on D129928

Our previous heuristic no longer works, so we now have to patch ICU directly to
support detecting the approximately sign.

Depends on D129930

This is no longer necessary after part 9.

Depends on D129931

Requiring at least ICU 70 allows to clean-up code in the next patch of this stack.

Depends on D129933

Number range formatting is stable in ICU 70, which means we can remove
MOZ_INTL_HAS_NUMBER_RANGE_FORMAT and MOZ_INTL_PLURAL_RULES_HAS_SELECT_RANGE.

The following bugs have been fixed in ICU 70, so we can remove the workarounds
for them:

Depends on D129934

https://unicode-org.atlassian.net/browse/ICU-21684 has been fixed in ICU 70,
so we no longer have to restrict the maximum positive exponent to 99'999. ICU
allows exponents up to 999'999'999, but formatting a number that large
requires multiple minutes in ICU. If we limit the maximum positive exponent to
9'999'999, we allow numbers much larger than just 99'999, while still
keeping the time needed to format the string at an acceptable level.

Depends on D129935

Hi André, these patches wedged Lando when they tried to land. We had to revert the repository to a known-good state to un-break things. Given that most of the people who know Lando are off until next week, can you please hold off on trying to re-land these patches until they're back? Feel free to reach out on Element to coordinate.

Flags: needinfo?(andrebargull)

Sure. Thanks for the info. I guess the conduit room is the best place to get in touch with the folks who know about Lando, right?


FWIW, here's the email I got from Lando about the failed request:

Your request to land D129937 failed.

See https://lando.services.mozilla.com/D129937/ for details.

Reason:
Aborting, could not apply patch buffer for 129921, 504440.
(255, b'', b"abort: abandoned transaction found!\n(run 'hg recover' to clean up transaction)")

Flags: needinfo?(andrebargull)

(In reply to André Bargull [:anba] from comment #18)

Sure. Thanks for the info. I guess the conduit room is the best place to get in touch with the folks who know about Lando, right?

Yes, exactly the right place. EST is going to be the best hours to discuss during with respect to who'll be around.

Pushed by rvandermeulen@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/ddb7978dfd7c Part 1: Update or remove ICU patches. r=platform-i18n-reviewers,dminor https://hg.mozilla.org/integration/autoland/rev/6826f1884d06 Part 2: Update in-tree ICU to release 70.1. r=platform-i18n-reviewers,dminor https://hg.mozilla.org/integration/autoland/rev/8f74f6afa9db Part 3: Update SpiderMonkey to Unicode 14. r=tcampbell https://hg.mozilla.org/integration/autoland/rev/d1b1ebc61bd3 Part 4: Update Gecko to Unicode 14. r=platform-i18n-reviewers,dminor https://hg.mozilla.org/integration/autoland/rev/c16119298a63 Part 4.2: Handle new script codes in GetCommonFallbackFonts. r=jfkthame https://hg.mozilla.org/integration/autoland/rev/c81dbbfa120b Part 5: Update language tag mappings. r=platform-i18n-reviewers,dminor https://hg.mozilla.org/integration/autoland/rev/ee994717b6cb Part 6: Update numbering systems. r=tcampbell https://hg.mozilla.org/integration/autoland/rev/9939757f5b22 Part 7: Update measurement units. r=tcampbell https://hg.mozilla.org/integration/autoland/rev/4635ffd59b2a Part 8: Handle new narrow-quarter symbol type. r=tcampbell https://hg.mozilla.org/integration/autoland/rev/9405f9389e29 Part 9: Add a separate UNumberFormatFields constant for the approximately sign. r=platform-i18n-reviewers,dminor https://hg.mozilla.org/integration/autoland/rev/a6cbb70bc84e Part 10: Remove approximately sign detection code. r=platform-i18n-reviewers,dminor https://hg.mozilla.org/integration/autoland/rev/31f4bb2c3ca6 Part 11: Update expected test results. r=platform-i18n-reviewers,dminor https://hg.mozilla.org/integration/autoland/rev/6d19255a7b6f Part 12: Bump minimum required ICU version to 70.1. r=platform-i18n-reviewers,dminor https://hg.mozilla.org/integration/autoland/rev/d65b061dd308 Part 13: Remove no longer needed ICU workarounds and conditional code. r=platform-i18n-reviewers,dminor https://hg.mozilla.org/integration/autoland/rev/c75ac4fa3429 Part 14: Allow a larger maximum positive exponent in ToIntlMathematicalValue. r=tcampbell https://hg.mozilla.org/integration/autoland/rev/a1b606f473a0 Part 15: Updating ICU requires a clobber. r=platform-i18n-reviewers,dminor
Regressions: 1742083
Regressions: 1742572
Regressions: 1751833
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: