Closed Bug 1714933 Opened 3 years ago Closed 3 years ago

Update our in-tree ICU to 69

Categories

(Core :: JavaScript: Internationalization API, enhancement)

Product:
Core
Component:
JavaScript: Internationalization API
Type:
enhancement

Tracking

()

Status:
RESOLVED FIXED
Milestone:
91 Branch
Tracking Flags:
Tracking Status
firefox91 --- fixed

People

(Reporter: anba, Assigned: anba)

References

Details

Attachments

(8 files)

Bug 1714933 - Part 1: Update ICU patches. r=tcampbell!
(deleted), text/x-phabricator-request
Details
Bug 1714933 - Part 2: Update in-tree ICU to release 69.1. r=tcampbell!
(deleted), text/x-phabricator-request
Details
Bug 1714933 - Part 3: Update numbering systems, measurement units, language tags, and time zone data. r=tcampbell!
(deleted), text/x-phabricator-request
Details
Bug 1714933 - Part 4: Update expected test results. r=tcampbell!
(deleted), text/x-phabricator-request
Details
Bug 1714933 - Part 5: Bump minimum required ICU version to 69.1. r=tcampbell!
(deleted), text/x-phabricator-request
Details
Bug 1714933 - Part 6: Remove guards around previous ICU draft APIs. r=tcampbell!
(deleted), text/x-phabricator-request
Details
Bug 1714933 - Part 7: Updating ICU requires a clobber. r=tcampbell!
(deleted), text/x-phabricator-request
Details
Bug 1714933 - Part 8: Update expected canonicalisation of "no". r=tcampbell!
(deleted), text/x-phabricator-request
Details

Also updates to CLDR 39 and brings various bug fixes and improvements.

Release notes: http://site.icu-project.org/download/69

Generate the WASI patch from bug 1706949 to apply cleanly on ICU 69.

Update to ICU 69.1 by running "update-icu.sh" with "maint/maint-69" as the target.

Depends on D116967

Re-run

./make_intl_data.py numbering
./make_intl_data.py units
./make_intl_data.py langtags
./make_intl_data.py tzdata

to update numbering systems, measurement units, language tags, and time zone data.

Also remove the "US/Pacific-New" link from otherICULegacyLinks() because it
is no longer needed. Output from ./make_intl_data.py tzdata with the link
still present:

Info: Link 'US/Pacific-New -> America/Los_Angeles' can be removed from otherICULegacyLinks()

Depends on D116968

Updating to CLDR 39 means a couple of format strings have changed, update the
expected results accordingly.

unicode-bcp47-locale-ids-language-mappings.js:
Replace "no -> nb" mapping with "tl -> fil" mapping, because CLDR 39 removed
the "no -> nb" mapping [1,2]. Use the "tl -> fil" mapping as the replacement,
because it is only present in CLDR, but not in the IANA language data registry.

[1] https://unicode-org.atlassian.net/browse/CLDR-2698
[2] https://unicode-org.atlassian.net/browse/CLDR-14493

Depends on D116969

Change the system requirement to ICU 69 in order to remove some conditional
code in the next patch.

Depends on D116971

udtitvfmt_formatToResult and udtitvfmt_formatCalendarToResult were both
promoted to stable in ICU 69, so we can remove the U_HIDE_DRAFT_API for this
code.

Depends on D116972

Depends on D116973

Blocks: 1636596
Pushed by andre.bargull@gmail.com: https://hg.mozilla.org/integration/autoland/rev/3de963fab6d6 Part 1: Update ICU patches. r=tcampbell https://hg.mozilla.org/integration/autoland/rev/e9f2a0d262c2 Part 2: Update in-tree ICU to release 69.1. r=tcampbell https://hg.mozilla.org/integration/autoland/rev/30e9aacecf56 Part 3: Update numbering systems, measurement units, language tags, and time zone data. r=tcampbell https://hg.mozilla.org/integration/autoland/rev/695f3bd8e875 Part 4: Update expected test results. r=tcampbell https://hg.mozilla.org/integration/autoland/rev/692101de3873 Part 5: Bump minimum required ICU version to 69.1. r=tcampbell https://hg.mozilla.org/integration/autoland/rev/40c329123b7c Part 6: Remove guards around previous ICU draft APIs. r=tcampbell https://hg.mozilla.org/integration/autoland/rev/2b4a65b9c1ac Part 7: Updating ICU requires a clobber. r=tcampbell

"no" is no longer canonicalised to "nb" in CLDR 39.

Flags: needinfo?(andrebargull)
Pushed by andre.bargull@gmail.com: https://hg.mozilla.org/integration/autoland/rev/85af7c63b9c5 Part 1: Update ICU patches. r=tcampbell https://hg.mozilla.org/integration/autoland/rev/f168a2d1d391 Part 2: Update in-tree ICU to release 69.1. r=tcampbell https://hg.mozilla.org/integration/autoland/rev/62bdba475376 Part 3: Update numbering systems, measurement units, language tags, and time zone data. r=tcampbell https://hg.mozilla.org/integration/autoland/rev/fca960ad267b Part 4: Update expected test results. r=tcampbell https://hg.mozilla.org/integration/autoland/rev/a4f4e4026174 Part 5: Bump minimum required ICU version to 69.1. r=tcampbell https://hg.mozilla.org/integration/autoland/rev/c7ccabae5816 Part 6: Remove guards around previous ICU draft APIs. r=tcampbell https://hg.mozilla.org/integration/autoland/rev/a42831b46643 Part 7: Updating ICU requires a clobber. r=tcampbell https://hg.mozilla.org/integration/autoland/rev/2f72875b300f Part 8: Update expected canonicalisation of "no". r=tcampbell

I don't understand why this failed. The error logs contain:

JS_Init failed: u_init() failed

which clearly indicates an error when initialising ICU. But when I push the changes to the try-server, I can't reproduce these failures: https://treeherder.mozilla.org/jobs?repo=try&revision=51fa58f8349acd6bea81d868db9ae215a41bb262. (And I can neither reproduce the failures locally.)

The errors for "Windows 2012 x64 debug" are also baffling to me:

[task 2021-06-14T20:54:58.624Z] 6:06.51 lld-link: error: relocation against symbol in discarded section: icudt69_dat
[task 2021-06-14T20:54:58.624Z] 6:06.51 >>> referenced by z:\task_1623694336\src\intl\icu\source\common\udata.cpp:698
[task 2021-06-14T20:54:58.624Z] 6:06.51 >>> ......\config\external\icu\common\udata.obj:(struct UDataMemory * __cdecl openCommonData(char const *, int, enum UErrorCode *))
[task 2021-06-14T20:54:58.624Z] 6:06.51 >>> referenced by z:\task_1623694336\src\intl\icu\source\common\udata.cpp:722
[task 2021-06-14T20:54:58.624Z] 6:06.51 >>> ......\config\external\icu\common\udata.obj:(struct UDataMemory * __cdecl openCommonData(char const *, int, enum UErrorCode *))
[task 2021-06-14T20:54:58.625Z] 6:06.51 lld-link: error: relocation against symbol in discarded section: icudt69_dat
[task 2021-06-14T20:54:58.625Z] 6:06.51 >>> referenced by z:\task_1623694336\src\intl\icu\source\common\udata.cpp:698
[task 2021-06-14T20:54:58.625Z] 6:06.51 >>> ......\config\external\icu\common\udata.obj:(struct UDataMemory * __cdecl openCommonData(char const *, int, enum UErrorCode *))
[task 2021-06-14T20:54:58.625Z] 6:06.51 >>> referenced by z:\task_1623694336\src\intl\icu\source\common\udata.cpp:722
[task 2021-06-14T20:54:58.625Z] 6:06.51 >>> ......\config\external\icu\common\udata.obj:(struct UDataMemory * __cdecl openCommonData(char const *, int, enum UErrorCode *))
[task 2021-06-14T20:54:58.625Z] 6:06.51 PLEASE submit a bug report to https://bugs.llvm.org/ and include the crash backtrace.

Later:
I've downloaded the patches from Phabricator and the ICU data file (icudt69l.dat) doesn't match what I have locally: It is an empty file on Phabricator! WTF.

Ah, okay. I think I understand what happened here: When the patches were backed out the first time, for some reason the ICU data file was replaced with an empty file: https://phabricator.services.mozilla.com/D116968?vs=450584&id=450646#change-c3mtSev48Kjf.
And when attempting to re-push the changes, the empty data file led to the errors which caused the second backout.

Flags: needinfo?(andrebargull)

Hi Atila,
the backout in https://bugzilla.mozilla.org/show_bug.cgi?id=1714933#c9 overwrote a binary file (see comment #14). I don't know which tools/programs are used to perform backouts, but maybe you know who can be informed that there seems to be an issue in the backout process which can lead to deleting files.

Thanks,
André

Flags: needinfo?(atila.butkovits)

This should be bug 1709608.

Flags: needinfo?(atila.butkovits)

Yes, thanks! That's exactly the issue observed here.

Pushed by andre.bargull@gmail.com: https://hg.mozilla.org/integration/autoland/rev/7bc79fc0a919 Part 1: Update ICU patches. r=tcampbell https://hg.mozilla.org/integration/autoland/rev/c73fe793c9b2 Part 2: Update in-tree ICU to release 69.1. r=tcampbell https://hg.mozilla.org/integration/autoland/rev/ec885d83c826 Part 3: Update numbering systems, measurement units, language tags, and time zone data. r=tcampbell https://hg.mozilla.org/integration/autoland/rev/ca2fccf42e3a Part 4: Update expected test results. r=tcampbell https://hg.mozilla.org/integration/autoland/rev/fc4205b8d565 Part 5: Bump minimum required ICU version to 69.1. r=tcampbell https://hg.mozilla.org/integration/autoland/rev/920e59de0c38 Part 6: Remove guards around previous ICU draft APIs. r=tcampbell https://hg.mozilla.org/integration/autoland/rev/a9def04a3236 Part 7: Updating ICU requires a clobber. r=tcampbell https://hg.mozilla.org/integration/autoland/rev/fd06b2ce2393 Part 8: Update expected canonicalisation of "no". r=tcampbell
Regressions: 1719150
No longer regressions: 1719150
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: