I am a GNU C Library (glibc) developer and I support the POSIX APIs for localization and internationalization.

I am also impacted by this change.

In the GNU C Library we are working hard to harmonize our locale data with CLDR. I think there is great value there. Having the glibc locales and Unicode/ICU/CLDR locales in sync means fewer surprise for our users. This can include adding new "formats" like those which CLDR has for dates. For example glibc just recently added alternative date formats to support two grammatical forms for month names (%OB and %ob), particularly for Slavic/Baltic languages that use genitive/nominative cases.

One of the benefits of glibc's POSIX API and locale implementation is that users can easily change their locale.

A good summary of the problem looks like this:

• Thunderbird has stopped allowing users to customize their date formats. Modifying the CLDR data is significantly more complex (expert level). Thunderbird tries to guess your appropriate CLDR locale by reading your system locale. There is no customization allowed beyond that.

• Previously Thunderbird using POSIX APIs and glibc's locale support. Users could build their own locales with their own formats for their system. For example I have my own 'en_US@ISO8601.UTF-8' that has ISO 8601 time for US English. It's really easy to do this.

This bug should really be retitled as "RFE: Add back support for system locales." which would allow users to switch back to their System Locales (POSIX APIs), and yet also allow the average user to stay on the acceptable default of ICU/CLDR (which I assume is used by Mozilla because it's a multiplatofrm library).

If you have any questions about the glibc APIs or other things, I'm happy to help, and improve them.

So far none of the extensions I have found solve the problem either. I don't consider it acceptable to switch to sv_SE locales.

Hi Carlos! Thanks for bringing your perspective. I'll try to respond to it.

In the GNU C Library we are working hard to harmonize our locale data with CLDR. I think there is great value there.

That's wonderful! I truly hope that we'll end up all using CLDR, since I see no value in duplication of efforts.

One of the benefits of glibc's POSIX API and locale implementation is that users can easily change their locale.

I feel I should make sure that we agree that such operation is an edge case and we don't expect majority of users to ever do that.
I'm not questioning the value of adding support for that, I'm just trying to verify if we agree on the relative priority of such feature vs. other features that we aim for (more on that later).

It's really easy to do this.

This continues to be a relative statement. Easy for whom? Someone able to understand the locale format, create patterns for skeletons correctly and write them to a file in a proper location? Sure. Is that "easy"?

I'd argue that nothing about internationalization data customization is easy, even on Windows or MacOS, and they try real hard to make it easy. But taking that into account, easy is when you open GUI of your OS Preferences and go to "Date & Time Format" and select "long date" to be "Day, Month, Year" rather than "Year/Month/Date" from a drop down.
Everything else is advanced.

This bug should really be retitled as "RFE: Add back support for system locales." which would allow users to switch back to their System Locales (POSIX APIs), and yet also allow the average user to stay on the acceptable default of ICU/CLDR

We could do something similar to what we do for Windows/MacOS, assuming glibc provides an API for such operation: https://searchfox.org/mozilla-central/source/intl/locale/windows/OSPreferences_win.cpp#119
https://searchfox.org/mozilla-central/source/intl/locale/mac/OSPreferences_mac.cpp#131

It should be fairly easy to provide the same API here: https://searchfox.org/mozilla-central/source/intl/locale/gtk/OSPreferences_gtk.cpp#134

If anyone is willing to write a patch, I can review it.

While I can appreciate consolidation, one thing that CLDR has done is turned what was a deliberate hack into a literal interpretation. en-DK was never meant to be a locale that had anything to do with Denmark. It was created to easily allow ISO-8601 date and time formats ofr English speakers. CLDR taking it literally as English Denmark has introduced undesirable characteristics such as number formatting and currency.

In the grand scheme of things, as an affected user I don't care as long as there is an eventual return of an easy way for me to get ISO-8601 dates and 24 hour time. I've implemented the en_SE.utf8 workaround by creaingan en_SE.utf8 system locale and modifying the envvars in the Exec lines of my user specific thunderbird.desktop file, but long term this should be offloaded to something more intuitive and accessible to less technical users.

For example I have my own 'en_US@ISO8601.UTF-8' that has ISO 8601 time for US English. It's really easy to do this.

I'm more than happy to try and create my own custom format but it seemed to be way above my technical level. That said if you can point me to a guide, documentation, video, podcast etc on how to do that I would seriously appreciate it.

(In reply to Zibi Braniecki [:gandalf][:zibi] from comment #104)

One of the benefits of glibc's POSIX API and locale implementation is that users can easily change their locale.

I feel I should make sure that we agree that such operation is an edge case and we don't expect majority of users to ever do that.
I'm not questioning the value of adding support for that, I'm just trying to verify if we agree on the relative priority of such feature vs. other features that we aim for (more on that later).

I don't see this as an edge case.

All users want to customize the display in their mail client.

Lack of time customization is simply a failing on behalf of the Thunderbird application.

However, I am not a Thunderbird developer, and so I don't get to make prioritization decisions, you need to do that.

This is high priority for me. The swapped en-US m/d/y will cause me to make mistakes and I can't tolerate that, so I'm going to probably switch back to Mutt, or another client after review.

It's really easy to do this.

This continues to be a relative statement. Easy for whom? Someone able to understand the locale format, create patterns for skeletons correctly and write them to a file in a proper location? Sure. Is that "easy"?

Easy for anyone. I'll show you. It was designed this way to be easy.

dnf install glibc-locale-source
su
cp /usr/share/i18n/locales/en_US /usr/share/i18n/locales/en_US@ISO8601
vim /usr/share/i18n/locales/en_US@ISO8601

Change 'd_fmt' to "%Y-%m-%d" (comments indicate which one is for %x)

Compile the locale:

localedef -f UTF-8 -i en_US@ISO8601 /usr/lib/locale/en_US@ISO8601.utf8

You're done. This is impossible to do with ICU/CLDR.

Even better is you can ship this in Ubuntu Universe or Fedora Copr.

Non-technical users don't even need to do anything but point at the repo, install the compiled locale, and set their LANG env var.

Again, this cannot be done with CLDR.

I'd argue that nothing about internationalization data customization is easy, even on Windows or MacOS, and they try real hard to make it easy. But taking that into account, easy is when you open GUI of your OS Preferences and go to "Date & Time Format" and select "long date" to be "Day, Month, Year" rather than "Year/Month/Date" from a drop down.
Everything else is advanced.

No, everything else is also easy. Just look at Windows.

Windows 10 allows you to directly[1] write the format exactly as you can do above for Linux, but they provide a UI for it (which is great).

[1] https://www.windowscentral.com/how-change-date-and-time-formats-windows-10

We had exactly this kind of flexibility with system locales, but ICU/CLDR does not provide that.

Windows 10 lets you write the patterns out exactly how you want to see them, so do Linux locales.

You could implement this in Thunderbird via ICU/CLDR APIs:

UnicodeString aPattern("GyyyyMMddHHmmssSSZ", "");
new SimpleDateFormat(aPattern, new DateFormatSymbols(Locale::getUS())
...

Example taken from: http://userguide.icu-project.org/formatparse/datetime

This bug should really be retitled as "RFE: Add back support for system locales." which would allow users to switch back to their System Locales (POSIX APIs), and yet also allow the average user to stay on the acceptable default of ICU/CLDR

We could do something similar to what we do for Windows/MacOS, assuming glibc provides an API for such operation: https://searchfox.org/mozilla-central/source/intl/locale/windows/OSPreferences_win.cpp#119
https://searchfox.org/mozilla-central/source/intl/locale/mac/OSPreferences_mac.cpp#131

It should be fairly easy to provide the same API here: https://searchfox.org/mozilla-central/source/intl/locale/gtk/OSPreferences_gtk.cpp#134

If anyone is willing to write a patch, I can review it.

I'll look at glueing this into the system locale. It looks like we already have "Preferences::GetBool("intl.regional_prefs.use_os_locales", false);" to fetch the preference which is somewhat mislabelled as "true" for Regional settings locale (means use the system defaults). I might propose another patch to change that, but we still have to honour that setting when picking a date format.

Please note that I forgot Mozilla's bugzilla has markdown syntax and I totally did not intend any large font highlighting like I did in my previous comment :-) Sorry.

All users want to customize the display in their mail client.

I think we disagree here and in the lack of any hard data on what percent of GUI mail client want to customize their date/time format patterns, we're trading opinions.

It was designed this way to be easy.

The moment you ask the user to touch the terminal and type commands, you exit the "easy" land for good.

Windows 10 allows you to directly[1] write the format exactly as you can do above for Linux, but they provide a UI for it (which is great).

Correct, from the GUI. And that includes validation of pattern syntax etc.

Please note that I forgot Mozilla's bugzilla has markdown syntax and I totally did not intend any large font highlighting like I did in my previous comment :-)

No worries, it was added very recently and we all get confused :)

I'll look at glueing this into the system locale.

Cool! I'm happy to see tighter integration into glibc, as long as it follows https://firefox-source-docs.mozilla.org/intl/locale.html#regional-preferences (which can be overridden by Thunderbird or/and by individual users using the pref).

(In reply to Zibi Braniecki [:gandalf][:zibi] from comment #109)

Cool! I'm happy to see tighter integration into glibc, as long as it follows https://firefox-source-docs.mozilla.org/intl/locale.html#regional-preferences (which can be overridden by Thunderbird or/and by individual users using the pref).

I agree 100%. This is what I meant by "Preferences::GetBool("intl.regional_prefs.use_os_locales", false);", which fetches the preferences from the store. The Preferences->Advanced->Date and Time Formatting radio controls actually set this to false (use ICU/CLDR) or true (try to use the OS information).

(In reply to Zibi Braniecki [:gandalf][:zibi] from comment #109)

All users want to customize the display in their mail client.

I think we disagree here and in the lack of any hard data on what percent of GUI mail client want to customize their date/time format patterns, we're trading opinions.

We must agree it's non-zero. Quite a few users are present here alone, but a simple Google search for the old en_DK hack shows there are many users who want this and those are just the few who took the time to make such a post. While many of those posts may not be specific to TB, I think it's safe to assume that if they do use TB, they'd expect their preferred format there too.

It was designed this way to be easy.

The moment you ask the user to touch the terminal and type commands, you exit the "easy" land for good.

Technical proficiencies vary wildly. But if TB no longer honors the glibc formats by any means short of personally applying a patch, I'd argue that it has been made extremely difficult for anyone. This sort of exclusion is the same reason I can't use GNOME: I don't always agree with the designers and want some choice to wiggle into a more comfortable setup that works for me. It's sad that something like this is now making me consider giving up many years of getting very comfortable with TB for an alternative MUA. First it's the loss of Google search, now this. I'm likely to start looking into kmail again -- I know it won't leave me hanging on something so basic as this. I'm losing faith in MSF. :(

It was designed this way to be easy.

The moment you ask the user to touch the terminal and type commands, you exit the "easy" land for good.

The mechanism for locale customization exists and works. It does not matter whether a user utilizes it via terminal or via some specialized user-friendly GUI.

Here are two fixes I came up with for this bug.

The first implements a new string preference named intl.locale.date_time.lc_time_override. If this preference is found, its value is used instead of the LC_TIME environment variable value. An example value for this preference is "root.UTF-8", which yields the ISO 8601 date/time format users want. This preference eliminates the need for fiddling with the platform's locale system or changing LC_TIME when starting Thunderbird.

The second implements a new string preference named intl.locale.date_time.pattern_override. If this preference is found, its value is used as the date/time pattern instead of the one that Mozilla comes up with. An example value for this preference is "y-MM-dd hh:mm", which yields the ISO 8601 date/time format users want. This preference provides very direct, granular control over the display of date/time values.

The code modifications can be found here:

https://gitlab.com/jeff_harp/thunderbird_60.4.0_date-time_mods

See the README.md file for more details.

I've allocated as much time to this as I can. If someone else wants to pursue these modifications further, please do so. I cannot.

@Jeff Harp
Thank you for your efforts.

I really like your second fix approach. To me it is very much like how I am used to fixing the issue in other apps and platforms. It should also work cross platform.

For me, your first approach does little for me. I want this same format system wide, so I already have locale and LC_TIME customized. It seems to apply only to Linux/Unix/BSD. I can see it's validity for others though.

I've used Thunderbird since its start, I have saved many (many) clients from buying 365 with this and Libre office. This is the only problem which irks me, smacks a bit of "we know best" it just seems right to allow the user to choose the format they prefer.
Any fix would need to affect the add ons, lightning is unusable for me because of this problem.
Regards

I'm just an end-user frustrated with this situation. I have also used Thunderbird since its start, moving during the years from OS/2 to Windows to Mac and now to Linux Mint. I'm from Finland, my native language is Swedish, but my systems have always been in English (US or GB). I have lived and worked in several European countries, now in France, and I want to have date and time formats according to the country where I live. This country/language/currency cocktail is maybe complicated for the Americans, but this is the way the world works. Please fix this.
Thank you.

And simply setting the advanced option "Date and Time Formatting: [x] Regional settings locale French (France)" doesn't work assuming that you have set your machine to a French locale? As I understand the bug, it's about a very specific use case which is not what you're describing.

No, it does not work. Thunderbird insists on en-US. The machine is set to regional French(France) and everything else respects that setting for date/time/hour/currency/decimal

But that's not due to the bug discussed here. Many Linux users use TB in en-US on their machine with a different locale and get localised date/time display. Do you see the option I mentioned, can you attach a screenshot?

(In reply to Jorg K (GMT+1) from comment #119)

But that's not due to the bug discussed here. Many Linux users use TB in en-US on their machine with a different locale and get localised date/time display. Do you see the option I mentioned, can you attach a screenshot?

Well, I would actually be curious about how so.

This is the exact reason I'm CC'd to this bug and others: My system is using en_US as locale, but I obviously want to use 24H format, as well as changing dates for example if possible.

Current format force user to some mental exercise every time he needs to read date, time and so on.

I changed the system settings, Preferences-Languages, Language to English-UK and the Region staying as French,France.
Now Thunderbird shows Regional settings locale as English (United Kingdom), dates show as DD/MM/YYYY and the time in 24-hour format. Much better like this.

Makes me think there is a problem of definitions: what is covered by the Language settings and what is covered by the Region settings. Linux, Mint, Cinnamon, or Thunderbird problem?

(In reply to Clément Lefèvre from comment #120)

Well, I'm not a Linux user, but I have seen felt 100 bugs on the issue and read a million comments.

We have Thunderbird developers in Finland and Slovakia who use a local locale and get localised display. If you use the en-US locale for regional display, all apps should should 12h AM/PM. If, unlike Windows, Linux doesn't allow the locale to be configured, why don't you used a less "mental exercise"-prone locale like en-GB. Also, please read the bugs quoted in comment #2. As I understand it, this bug here is about LC_TIME=en_DK.utf8 and nothing much else.

(In reply to henri.passerieux from comment #121)

I changed the system settings, Preferences-Languages, Language to English-UK and the Region staying as French,France.

I still don't understand why you don't set absolutely everything to French. Then TB would do "regional settings locale French (France)". Is that not what you want?

At least on Windows, TB follows format for a selected region, I don't know that the regions per se is? You mean location?

Obviously the Linux version of Thunderbird does not pick up the system setting that Linux Mint calls Region, only the system language setting. And I do want to keep the system language as English. Now it's English UK and I'm fine with that. Next time I know to select that already when installing.

Obviously the Linux version of Thunderbird does not pick up the system setting that Linux Mint calls Region, only the system language setting

We do some special customizations for the old Unity from Ubuntu. I would accept a patch that extends that to cover Mint preferences if anyone is willing to write a patch :)

Any chance you consider Jeffs patch? Being able to define one's own date-format string would be very helpful.

not directly, no. The patch would need generalization (for example, it provides a single pattern, while we need patterns per style - narrow, short, medium, long, full etc.)

I'd also like to see how it fits into the conversation in https://github.com/tc39/ecma402/issues/190

To sum it up - in principle, I'm not against extending our platform capability to fine-tune intl settings, and I recognize that date/time patterns are particularly sensitive area to customization.
But I don't think we should be solving it on the Gecko level by writing shortcuts through the system to satisfy some particular UI widget.

To make it actionable: If someone is willing to write a patch which provides hardcoded custom patterns for date, time and date-time styles (narrow/short/medium/long/full).

I imagine such API to lead to us being able to add UI similar to what MacOS and Windows do [0].

It should likely be a new API in intl/locale, named maybe I18nSettings, which would manage setting and retrieving custom overrides for date/time patterns (and maybe first day of the week, like Windows does?) if they're set, or use data from locale / os preferences when not.

Happy to mentor, but I won't have time to work on that anytime soon.

[0] https://www.windowscentral.com/how-change-date-and-time-formats-windows-10

Bibi wrote in Comment #24

• we will match the locale that your OS returns, but we'll take the values
  from CLDR since we know we have all the tables for each locale that we need.

(Just to clarify - by "locale" do you mean the locale-name ?)
But if you don't use the values from the POSIX locale, why do you require
that the locale-name be valid as a POSIX locale on my system?
All that matters is that it is a valid CLDR locale.

In particular, I would be very happy to use the CLDR "root",
see http://demo.icu-project.org/icu-bin/locexp?_=root ,
as its dates are in order Y/M/D and the times are 24-hr.
Why not simply let us specify this name, "root", directly as a preference?
If a name is specified and it is not a valid CLDR locale, then you
can use the current procedure as a fall-back.

(I am asking this because I have not been able to figure out how to
make "root" a valid POSIX locale name on my system - Linux Mint.
The recipe in Comment #59 didn't work for me.)

(In reply to Carlos O'Donell from comment #107)

Easy for anyone. I'll show you. It was designed this way to be easy.

Carlos, I'm trying to follow your instructions but for me they are anything but "easy":

dnf install glibc-locale-source

I think this is highly distribution-dependent. I have no idea what dnf is but it doesn't exist on my system (openSUSE). I'm guessing that this command is supposed to invoke some package manager that installs the glibc locale sources. The source package turns out to be named something completely different on openSUSE -- it's called glibc-i18ndata.

vim /usr/share/i18n/locales/en_US@ISO8601

Change 'd_fmt' to "%Y-%m-%d" (comments indicate which one is for %x)

For me, the value of d_fmt in en_US is "<U0025><U006D><U002F><U0025><U0064><U002F><U0025><U0059>", which is completely opaque. All the other variables in this file, and every other file I've examined, are defined with similar escape sequences. Most files don't have explanatory comments (not even ones that simply unescape the values). It's not obvious whether I'm even allowed to replace an escaped value with an unescaped one, nor how to easily automatically convert an unescaped string to an escaped one. (I'm sure as hell not going to look up every single character in a Unicode table so that I can type it in manually, and writing a program to do so seems to be about as much effort.) In fact, a less technically sophisticated user wouldn't even understand the fact that the values are escaped.

Compile the locale:

localedef -f UTF-8 -i en_US@ISO8601 /usr/lib/locale/en_US@ISO8601.utf8

You're done. This is impossible to do with ICU/CLDR.

Well, I tried copying and pasting some existing key-value pairs from other locale files to build a new one, en_CA@ISO8601. The localedef command seemed to compile my new locale, but (for reasons probably mentioned elsewhere in this conversation) Thunderbird doesn't seem to want to use it. When I run it with LC_TIME=en_CA@ISO8601.utf8 and/or LANG=en_CA@ISO8601.utf8, all the dates and times in the mail list show up in the usual en_CA format (i.e., with YYYY-MM-DD dates but hh:mm a.m./p.m. times).

(In reply to Len Weisberg from comment #128)

But if you don't use the values from the POSIX locale, why do you require
that the locale-name be valid as a POSIX locale on my system?
All that matters is that it is a valid CLDR locale.

At least for me this is not required. If you start Thunderbird with "LC_TIME=root thunderbird" it uses CLDR’s “root“ locale time and date format, even if it does not exist as a installed system locale. There’s a Gtk-Warning about fallback to C locale, but this does not seem to have impact, as Thunderbird ignores it anyway.

As a hotfix, I overrode the system desktop file for Thunderbird by copying it to ~/.local/share/applications/ and put "env LC_TIME=root " before the binaries in all Exec lines. No tinkering with the POSIX locales.

(tested on Arch Linux and Thunderbird 60.6.1)

(In reply to simon.marquardt from comment #130)

(In reply to Len Weisberg from comment #128)

But if you don't use the values from the POSIX locale, why do you require
that the locale-name be valid as a POSIX locale on my system?
All that matters is that it is a valid CLDR locale.

At least for me this is not required. If you start Thunderbird with "LC_TIME=root thunderbird" it uses CLDR’s “root“ locale time and date format, even if it does not exist as a installed system locale. There’s a Gtk-Warning about fallback to C locale, but this does not seem to have impact, as Thunderbird ignores it anyway.

As a hotfix, I overrode the system desktop file for Thunderbird by copying it to ~/.local/share/applications/ and put "env LC_TIME=root " before the binaries in all Exec lines. No tinkering with the POSIX locales.

(tested on Arch Linux and Thunderbird 60.6.1)

If all you want is YYYY-mm-DD HH:MM date format in Thunderbird, then yes you do not need to tinker with the POSIX locales. If you want the rest of the system to have the same date time format then you do have to tinker with the POSIX locales because that is how the system date time format is defined.

The change in Thunderbird caused us to have to change our workaround to ensure the LC_TIME variable value matched the new approach using a CLDR locale name.

We have to keep in mind that "root" is not really a locale per se in CLDR but a fallback, base, or reference. If it is decided that "root" is unnecessary in CLDR, then this workaround will likely stop working in Thunderbird. But the system date time format will continue to be as we desire no matter what we choose as a name.

I am asking the CLDR folks for help with all of this. Please see my new ticket:
https://unicode.org/cldr/trac/ticket/12027
"CLDR should support (some variants of) ISO-8601 date formats."

Mozilla developers, Thunderbird is the only app on my system that screws up the date. This isn't a case of there not being an API or something. The fault lies with Mozilla developers. Thunderbird should at least display date & time format similar to how other apps are handling this. Every other app on my system keeps 24-hour time and displays the date according to GNOME's Region & Language setting.

Can you please mark this for priority bug fixing?

In the meantime, is there any about:config setting or extension that can remedy this situation? Date and time is important when dealing with emails.

Screenshots of the problem:
https://imgur.com/a/XlqdXxV

Every other app on my system keeps 24-hour time and displays the date according to GNOME's Region & Language setting.

Can you provide Steps to Reproduce for when Thunderbird doesn't follow your selection of 12/24h clock in Gnome? Because it should.

Please, also attach the Internationalization section of your about:support.

Thank you!

source of where we look up Gnome 12/24 clock settings - https://searchfox.org/mozilla-central/source/intl/locale/gtk/OSPreferences_gtk.cpp#144

I'm wondering if that's one of the places in UI where Thunderbird doesn't use Gecko APIs that respect Gnome's 12/24h.

Can you provide Steps to Reproduce for when Thunderbird doesn't follow your selection of 12/24h clock in Gnome? Because it should.

1. Run thunderbird -profilemanager
2. Set up a new profile
3. Set up an email account
4. Look at the date column values

Actual behaviour: The values are shown in a weird format: "day of month without leading zero/month with leading zero/two-digit year, hour without leading zero:minute with leading zero [am|pm]", for example "1/05/19 6:09 am".

Expected behaviour: Dates should show up either as printed by the relevant utility:

$ date
Wed May 29 18:54:02 NZST 2019

or as the by now universal coarse-grained ISO format of "YYYY-MM-DD hh:mm Z".

Internationalisation & Localisation settings:

Application Settings
Requested Locales 	["en-NZ","en-US"]
Available Locales 	["en-GB"]
App Locales 	["en-GB","und"]
Regional Preferences 	["en-NZ"]
Default Locale 	"und"
Operating System
System Locales 	["en-NZ"]
Regional Preferences 	["en-NZ"]

The values are shown in a weird format:

I'm afraid there are multiple separate issues conflated in this bug into one.

In your case, you want to use en-NZ, and we format it according to Unicode/CLDR convention for en-NZ locale:

(new Date()).toLocaleString("en-NZ");
"29/05/2019, 12:37:11 am"

So, in your case, assuming he locale selection looks reasonable (we only have en-GB language pack, so the strings will be in en-GB, but your intl formatters use en-NZ), I can only recommend you reporting to CLDR that en-NZ date time format is broken.

Greg had, I believe, a different problem. He wanted his 12/24h setting to be respected, and I pointed out that our code does, in fact, respect 12/24h manual override from Gnome.

I'm afraid that everyone who doesn't like how Tb displays their datetime format come to this bug stating "just make it work", but each one may be describing a different problem solvable in a different way.

Same as comment #133, My clock is on 24H in Gnome while locales are en_US, and it shows time as 12H (and date as US format, which I would like to change too).

Here's i18n part of about:support:

Internationalization & Localization

  Application Settings
  Requested Locales: ["en-US"]
  Available Locales: ["en-US"]
  App Locales: ["en-US","und"]
  Regional Preferences: ["en-US"]
  Default Locale: "und"

  Operating System
  System Locales: ["en-US"]
  Regional Preferences: ["en-US"]

So according to this thread, thunderbird should ignore whatever the system locales, read the name of a locale out of LC_TIME or whatever, and then look up stuff in the CLDR.

So why must the contents of LC_TIME be a real locale on the system?
I would think I should be able to run LC_ALL=root thunderbird and get usable dates as it just looks up what the CLDR claims root is.
But that doesn't work, root needs to be a real locale to get usable dates.

Any idea why it needs to be a real locale? Can that be fixed so thunderbird is completely separate from system locales, and maybe it's less confusing as to why thunderbird only kinda respects what locale is set?

Looking back closer, this was noted by Jonas Witschel months ago: https://bugzilla.mozilla.org/show_bug.cgi?id=1426907#c19

So steps(for debian at least) all in one place for the next person unfortunate enough to encounter this thread:

sudo cp /usr/share/i18n/locales/en_DK /usr/share/i18n/locales/root
echo 'root UTF-8' | sudo tee -a /etc/locale.gen
sudo locale-gen
sed 's/Exec=\/usr\/bin\/thunderbird/Exec=env LC_TIME=root \/usr\/bin\/thunderbird/' /usr/share/applications/thunderbird.desktop > ~/.local/share/applications/thunderbird.desktop 

Thunderbird 60.8.0 (from Debian).

I think, the new Thunderbird has a more serious problem with locales. I use my own custom locale, named "en_EU", which defines the date formats in the "European style" (German and Russian at least) but with English months and days names (like "Sun, 28 Jul 2019" and "28.07.2019"), and the 24-hour time format with colons (like "13:00" and "13:00:00").

So I have LC_TIME=en_EU.utf8 set, and all other programs that I use can understand this and show dates/times in my custom format. Thunderbird 45 also worked perfectly with it. But the current Thunderbird version apparently cannot use it at all, even though LC_TIME=en-DK does work (but I do not need that, I want my own format).

Can it be made to honor the actual system locale, whatever it is? Or at least provide customizable date/time formats, if the proper system-locale usage, which was working fine in the previous versions, magically became impossible in the past couple years?..

Can it be made to honor the actual system locale, whatever it is?

As long as it matches one of the CLDR ones.

Or at least provide customizable date/time formats,

I'm open to this, but it requires quite some work and noone has committed to put that work in.

(In reply to Marc from comment #141)

Honour the actual locale, what does that mean any more? Yes there are some local anomalies as to the format of the date, and yes these can be offered as the default setting, BUT, and this is the whole point of free software, the end user should be able to override the suggestion and use whatever they want. Of course ISO dates will eventually become the norm (if only because it is the current Chinese format:) and as a programmer this is my preferred format, so ISO dates must be available - not a great problem really as most (if not all) programming languages use this format anyway.

As I am approaching retirement, I may dig into the code and see what can be done, although there is plenty of good code out there already as most of the large open source projects do this anyway.

This ticket was opened 2017-12-22 18:26 PST (yes, ironically, that is
how bugzilla formats the date in the tooltip under "2 years ago",
actually more like 1.7 years ).

I appreciate that this bug has wandered into many issues and also that
there may be no one available to work on it. But I would just like some
help with working around what I believe is the main problem:
I would like help in cajoling TBird to use CLDR's "root" locale time and
date format. I feel certain that I am by no means alone in wanting this.

Some workarounds have been presented here, (eg in Comment #59), but they
don't seem to work for me. I am guessing that the key thing missing is a
file to be installed as a "root" locale. (I also have a hunch that
this file could be very simple if it is intended only to act as the
value of LC_TIME for running TBird.)
Could one of the experts here please supply this "root" locale file
and fill in and/or correct the following outline:

As root user:

1. copy the simplified file (found at ...) to /usr/share/i18n/locales/root
2. run: echo 'root.UTF-8 UTF-8' >> /etc/locale.gen
3. run: locale-gen
   As ordinary user:
4. set TBird Prefs -> Advanced -> "Date and Time Formatting"
   to ...
5. quit TBird and restart with LC_TIME=root.UTF-8

Does this seem feasible? Can we get some help with this?
(BTW, I am using Linux Mint 19.1 - based on Ubuntu 18.04)

Thanks much,
-Len

@len what you detail looks about right. The workaround shown in comment #138 was the one that got me to happiness. Adapted to Fedora, it went like this:

sudo dnf install -y glibc-locale-source
sudo cp /usr/share/i18n/locales/{en_DK,root}
sudo localedef -v -i root root
sed 's/Exec=thunderbird/Exec=env LC_TIME=root thunderbird/' \
       /usr/share/applications/mozilla-thunderbird.desktop \
       > ~/.local/share/applications/mozilla-thunderbird.desktop

As for TBird Prefs -> Advanced -> "Date and Time Formatting", I see "Regional settings locale: root" selected. I don't recall what I saw there or had set before doing the above. Best of luck!

Well, I see now that this workaround really is as simple as described
in Comment #138 (and others). And no need for a special locale file.

The only weak link now is the stability of the CLDR's "root" locale.
There has actually been some activity on the ticket I filed with
the CLDR folks (see Comment #132), but I have no idea what it means
in terms of anything concrete.

(In reply to Zibi Braniecki [:zbraniecki][:gandalf] from comment #140)

Can it be made to honor the actual system locale, whatever it is?

As long as it matches one of the CLDR ones.

Every OS provides standard APIs for locales and allows end-users to customize these locales to their personal needs. What was the purpose of breaking this approach, which has been working well for many years, in favor of some third-party libraries with incomplete hard-coded stuff? Did it solve any problems? So far I can only see that is has created problems for many users. And worst of all, these problems apparently cannot be circumvented by any reasonable means (I do not consider custom-patching the source code and recompiling everything as a viable solution).

Or at least tell us how to define a custom CLDR locale, if Thunderbird developers completely refuse to use the standard OS locale APIs, as they did in previous versions.

Or at least provide customizable date/time formats,

I'm open to this, but it requires quite some work and noone has committed to put that work in.

There was some patch in comment #113.

What was the purpose of breaking this approach

Gecko is not an OS. And unfortunately we do not provide a full UI/UX for customizing the locales to personal needs.

in favor of some third-party libraries with incomplete hard-coded stuff?

Please, educate yourself on the topic you're writing an opinion on.

Did it solve any problems?

Yes. It's been a very important and large scale refactor of the ecosystem which allowed us to remove large amount of unmaintained code and simplify the maintenance of the intl module. If you are curious about more, please, read the changelog, or my blog posts from last years about it.
If you did not have time or interest, assume your ignorance in the topic and avoid claiming knowledge.

So far I can only see that is has created problems for many users.

Your visibility of the impact is limited. How often do you triage the Intl module?

And worst of all, these problems apparently cannot be circumvented by any reasonable means

They absolutely can. It just takes time to write customization overlays, or platform bindings. If you have interest in that, I'm happy to mentor.

Or at least tell us how to define a custom CLDR locale

I doubt there's a concept of "custom CLDR locale", but feel free to read the docs and find out yourself - http://cldr.unicode.org/

There was some patch in comment #113.

That patch would be a good start, but requires a bit of work to get it into making it applicable for upstream (see Comment 127 ).

I'm tempted to restrict comments in this bug. The signal to noise ratio is declining and developer conversation about how to fix this bug vs. "I want X" as well.
Since it's in TB I'll leave it open to TB triage owners to decide, but I'm likely going to stop responding since I am repeating myself to people who seem to have enough interest to voice their opinion, but not enough to dive into the problem domain.

Comment #144 This is fantastic, thank you @jflorian

My gnome LC_TIME locale is set to "en_DK.UTF8", which should produce time format YYYY-MM-DD HH:MM

Thunderbird is the only application that doesn't honor this. Is there a way to get it honored until this bug is fixed proper? Can someone post/repost a way to fix this now (if a way exists)? Thanks!

(In reply to Greg from comment #150)

My gnome LC_TIME locale is set to "en_DK.UTF8", which should produce time format YYYY-MM-DD HH:MM

Thunderbird is the only application that doesn't honor this. Is there a way to get it honored until this bug is fixed proper? Can someone post/repost a way to fix this now (if a way exists)? Thanks!

You can find this information by reading the comments above, but since I have just spent a considerable time reading them all myself and more people might start looking at the latest comments first, here is the gist:

Thunderbird still honors your LC_TIME but it has changed how its value is interpreted; in the past Thunderbird used the locale definition found under that name on your system, now it looks up data like the date format in the Common Locale Data Repository (CLDR), which also has a locale named "en_DK" but with a different date and time format. This format is what you see now in Thunderbird.

It sounds like the general change in locale resolution has made things easier in terms of code maintenance and cross-platform consistency; and since CLDR is considered a reputable source for the locale definitions, the issue of this ticket is not actually considered a bug by the developers, or at least not one in Thunderbird. It would have to be fixed in the CLDR.

Now, for a solution that can be applied by the users right now, the best way seems to be setting LC_TIME to a pseudo locale called "root", which is provided by the CLDR and actually uses ISO 8601 date and time format. "root" does not exist as a locale on your system but you can just create it as a symlink to en_DK. See comment 59 for detailed steps.

(In reply to Eric Toombs from comment #102)

In case it's too hard to find, the post with title "WORKAROUND" is here:
https://bugzilla.mozilla.org/show_bug.cgi?id=1426907#c88

Eric, you are referring someone who says he is using openSUSE to a post containing an Arch-specific workaround. That workaround doesn't work on openSUSE.

(In reply to Bernhard Groll from comment #151)

You can find this information by reading the comments above, but since I have just spent a considerable time reading them all myself and more people might start looking at the latest comments first, here is the gist:

And Bernhard, the workaround from Comment 59 that you recommend is also distribution-specific. As others have already reported here, it doesn't work on Fedora and it doesn't work on openSUSE. The commenter you are responding to doesn't mention what system he is using, so it's impossible to say whether your recommendation will help him.

Using this Bugzilla issue to document workarounds for all the different systems supported by Thunderbird is not particularly efficient. It's very hard for users to find these workarounds in this gargantuan thread, and it is very hard for them to determine whether or not any given workaround is applicable to their system. Besides this, devising, critiquing, and improving these workarounds is probably not an intended use of the issue tracker. It would be better if we could set up an editable list of workarounds, and an associated discussion on same, on a wiki somewhere, and then direct users to it when they ask for a workaround. But I'm not sure if there's any existing wiki that would be appropriate for this; the most appropriate ones I know of, like the official Arch and openSUSE wikis, are all distribution-specific.

I should clarify that when I say "that workaround doesn't work", I'm not referring to the general idea of setting up a new locale, but rather to the detailed series of steps for creating and registering such a locale. Different *nix distributions put the locales and associated configuration files in different places, and use different commands for compiling/registering/enabling these locales. Very experienced users may be able to read, say, a set of Debian-specific instructions and, by consulting the relevant documentation for Debian and for their own distribution, and with a little (or a lot of) trial and error, successfully adapt those instructions to their own system. But most people coming to this thread probably lack the knowledge, time, or patience to do this. And among the rest, not all of them are going to come back and post their adapted workaround as a clear and detailed series of instructions.

Also affected.

On older Thunderbirds, LC_TIME=en_DK.UTF-8 would display a weird time format using . as hour/minute separators, similar to what Eric Toombs experienced in comment 84.

On Thunderbird 68.2.2, I am getting the hybrid date format experienced by everyone in this bug report. sv_SE.UTF-8 only partially solves the problem, as long date formats with the weekday would use ths Swedish name for the day.
However using LC_TIME=en_DK (without .UTF-8), which is NOT installed on my system, makes Thunderbird complain:

Gtk-WARNING **: 16:14:12.189: Locale not supported by C library.
Using the fallback 'C' locale.

but use the correct ISO-8601 date format regardless! Very weird. Indeed, Thunderbird must have some terrible broken internal mangling of locales to come to that conclusion.

Anyway, LC_TIME=en_DK works.

@haarp thanks for the hint. On Linux, adding LC_TIME=en_DK in /etc/default/locale did work for me also but only if en_DK was not installed. It also worked with LC_TIME=C.UTF-8 which I think is a better solution since it removes the warning "Using the fallback 'C' locale." when starting Thunderbird and it will not produce a warning when you run the command "update-locale". It will also work even if you have en_DK locale installed or not. It looks like en_DK support was removed in Thunderbird?
In Firefox, I noticed an interesting behavior that may be related to this Thundebird bug. Usually, if we use an English version of Firefox, and want to use an HTML input type=time with a 24-hour clock instead of AM/PM (https://developer.mozilla.org/en-US/docs/Web/HTML/Element/input/time), we need to set intl.regional_prefs.use_os_locales to true in about:config. It will follow the operating system locale and also use a date like DD/MM/YYYY instead of MM/DD/YYYY (https://developer.mozilla.org/en-US/docs/Web/HTML/Element/input/date).
However if we use LC_TIME=C.UTF-8 (or LC_TIME=en_DK without installing en_DK locale), we don't need to set intl.regional_prefs.use_os_locales to true, the operating system locale is selected.
I noticed a weird behavior also with the preference intl.locale.requested in Firefox. If we set it to "en" (without quotes), while using LC_TIME with en_DK.UTF-8 or en_IE.UTF-8 for example, we don't need to set intl.regional_prefs.use_os_locales to true, in order to select the operating system locale.
When using LC_TIME=C.UTF-8 (or LC_TIME=en_DK without installing en_DK locale), we can set intl.locale.requested to "en" in order to use the web browser locale instead of the operating system locale.

Here's a patch that implements a more robust, two-tiered solution to this problem. It significantly improves upon what I proposed in my earlier comment (113), and takes into account the concerns Zibi raised in comment 127.

This approach provides simplicity for users who just want the ISO formatting (as associated with the root locale), but also provides a way for advanced users to customize every aspect of date/time formatting to address their particular needs.

All the preferences have a common prefix of "intl.locale.date_time.pattern_override".

The first tier is a simple boolean preference named "root". When true, date/time values are formatted based on the CIDR root locale. This provides the ISO formatting others have requested.

The second tier is a boolean preference named "custom", along with 11 string preferences that hold patterns (7 for date, 3 for time, and 1 for the connector) that are used to implement fully custom date/time formatting. The default values for the 11 string preferences are based on the CIDR root locale and hard coded values found in DateTimeFormat.cpp.

When both of the new boolean preferences are false, date/time formatting takes place normally, based on the user's platform locale.

This patch modifies 3 files:

intl/locale/DateTimeFormat.cpp
intl/locale/DateTimeFormat.h
modules/libpref/init/all.js

These changes are outside the "comm" folder, so they impact Firefox and other projects as well. I don't know what that means in terms of how one would go about submitting a patch like this for formal consideration, as I have no prior experience dealing with Mozilla as a developer.

For what it's worth, the patch works nicely with Thunderbird 68.4.1 and Firefox 74.0.1, the versions available from the Ubuntu Linux 18.04 LTS repositories, and which I use daily.

Thank you for the patch Jeff! I'll review it shortly!

This patch looks really good!

== Big picture ==

Unfortunately, the code you're reworking is really old, and out of date with how Gecko intl works today.
If it was only about altering code, I'd be ok with updating the code as is, and keeping the refactors for later but because your patch introduces preferences, and preferences are tricky to maintain/deprecate, I'd like to iron out the code before we introduce prefs.

To put things in context - I'm working on a project that is meant to introduce Rust backed DateTime formatting, which will hopefully replace ICU uses in long term and give us much better performance.
The API is going to be fairly well aligned with ICU and ECMA402 tho, so all alignments we can do today to get this class more in line with those modern APIs will serve us well when we switch to Rust.

== Patch comments ==

Comments:

1. I think it would be better if you pulled root patterns from ICU rather than hardcoding them in CPP code.
2. I think the particular custom prefs should not be set by default at all. If the user wants to set one, they would add it (or the UI in Preferences would add it).
3. I'm not sure if there's a value in the boolean flags. A set preference is a signal in itself. Therefore, I think I'd prefer to limit that to one pref per option. I understand that the ergonomics won't be perfect, but we are addressing an edge case, so I think it's okay to make such tradeoff.
4. Unless there's a strong use case, I'd really love to avoid having to also support overriding the connector.
5. I think we should revisit what aDateFormatSelector and aTimeFormatSelector we support.

The fifth point warrants a separate bug I believe, but I'm going to provide the rationale here.
We currently support 13 different selectors, and many of them are obsolete or misguided. I'd like us to coalesce around the dateStyle/timeStyle proposal - https://github.com/tc39/proposal-intl-datetime-style - or a subset of it (long/short for example).

This would give us a number of benefits:

• matching consistent styles between JS and C
• matching Ecma402 standards
• good foundation for in-OS hooks so that we can take those styles from OS later

I can see us starting with none/short/long for both date and time for now, and that would narrow it down to 6.

Here's an example of how we do it for localization needs. Those two pieces could be merged then - https://searchfox.org/mozilla-central/source/intl/l10n/FluentBundle.cpp#267-312

As for the others:

• kDateFormatMonthLong and kDateFormatWeekday are not patterns. They're symbols. We should have DateTimeFormat::GetSymbol that uses https://unicode-org.github.io/icu-docs/apidoc/released/icu4c/classicu_1_1DateFormatSymbols.html to return weekday/month in styles long/short/etc.
• kTimeFormatSeconds and kTimeFormatNoSeconds should be converted to kTimeFormatLong and kTimeFormatShort
• kDateFormatYearMonth is not used in Firefox and I hope neither is in Thunderbird so we should be able to just remove it.
• kDateFormatYearMonthLong is used in one place in Firefox. I'd like to suggest that we either introduce DateTimeOptions bag like ECMA402 does, or, for simplicity, start with a variant of the method that take a skeleton string instead of styles. A skeleton is a semantically concise version of an options bag, so the caller in Firefox's NavHistory would do DateTimeFormat::FormatPRExplodedTime("yyyyMMMM", &aTime, monthYear); since they want this level of precision.

One we have this, we would introduce the following four prefs:

intl.locale.date_time.pattern_override.date_long
intl.locale.date_time.pattern_override.date_short
intl.locale.date_time.pattern_override.time_long
intl.locale.date_time.pattern_override.time_short

If those values are not set, use Gecko logic. If they're set to root, use ICU's root patterns, if they're set to a non-root string, use it as a pattern.

== Proposal ==

What does it mean for this patch?

If this big-picture proposal sounds good to you, I suggest that we:

1. File a bug to extend the API with GetSymbol and skeleton
2. File a bug to clean up aDateFormatSelector and aTimeFormatSelector to use the new methods
3. Update the patch in this bug to use the new conventions from (1) and (2)

How does the plan sound to you Jeff?

It's exciting to see what looks like major progress toward solving the
problem addressed by this bug. Just a reminder of a CLDR ticket I filed
that could perhaps someday mean a replacement for the special name "root".
But do note that the Unicode website has been moved and reorganized
and my ticket is now at:
https://unicode-org.atlassian.net/browse/CLDR-12027
"CLDR should support (some variants of) ISO-8601 date formats."

(In reply to Zibi Braniecki [:zbraniecki][:gandalf] from comment #158)

How does the plan sound to you Jeff?

Thanks Zibi. I appreciate your quick and thorough response.

I honestly was viewing the submission of my patch as an endpoint, rather than a starting point. I modified the current code to address what I (and others) consider to be a serious bug, whether or not Mozilla considers it a bug. I was doing it as a user who happens to be a developer, not because I was seeking to become more involved in contributing to Mozilla code development. And I was hoping the patch would be accepted with minimal changes so that users could benefit from it sooner, rather than later.

I stopped actively developing software about two years ago to become primary caregiver for my elderly parents. That's a 24/7 "job" allowing little time for anything else. And right now my father is critically ill in hospital with a systemic bacterial infection and fighting for his life.

So I'm unable to contribute further at this time. I hope someone else will jump in and pick up where I left off. If I'm able, I may work on this again in the future, but I currently have no idea when that might be.

So I'm unable to contribute further at this time. I hope someone else will jump in and pick up where I left off. If I'm able, I may work on this again in the future, but I currently have no idea when that might be.

Thank you for your response! I completely understand!

I wasn't asking for you to take on fixing this, rather for your opinion on my proposal on how to fix it. :)
If we agree on the approach and an end point, I can file bugs and hopefully either find time to work on it myself, or flag the bugs as good first bugs for other contributors to take!

Zibi -> Please-Please tell me this is still proceeding !
We must stop this madness of setting time/date format based on locale/language preferences or hooks to a quagmire of internal OS settings.
In the age of ISO the idea that my mailbox time/date formatting must conform to UK/English, French/Canada, etc. is beyond frustrating.
And sadly with the multitude of Unix derivatives adopting all forms of non-standard, personalized desktops you can't depend on the time displayed on the desktop or many of the former default OS/Desktop system-time hooks being correct or even existing. SystemD, Cinnamon, etc. I have wandered the internet and found a myriad of complaints and a lot of unsuccessful answers.
If there is any way to create an interface where clients can select their own preferred time/date format options, check the result and then possibly adjust for time zone offset that would be great. I would then use that interface for other custom applications as well.

• sign me up for testing !!
  Another annoyance I did find a "fix" for but should be an interface selectable option; the "Date" field for mail received today only shows the time. Why ?? Why ??

Zibi -> Please-Please tell me this is still proceeding !

As you can see, there's no assignee for this bug. That means that it is not proceeding, but it is available for grabs.

There's some conversation on privacy of Intl APIs in https://github.com/tc39/ecma402/issues/443 which may result in us closing the gap between date/time settings we use for internal APIs and expose to the Web. If that will be the case, we'll be able to expose your Windows/MacOS/Linux/Android internationalization preferences to the Web APIs, which would help a lot.
But that's a slow process unfortunately, and we currently do not have available engineers to work on that :(

Does anyone know how to change this page ? "http://kb.mozillazine.org/Date_display_format"

It contains information about the no longer working workaround, and it would be more honest if there was a redirect to this bug.

This page was last modified 13:44, 13 February 2017. This page has been accessed 476,809 times.

As a user,

Bugzilla is not intended for "as a user" type of communication. Please, read the Bugzilla Etiquette - https://bugzilla.mozilla.org/page.cgi?id=etiquette.html

If you're interested in continuing work on the patch and proposal from comment 158, feel free to take this bug, it's currently unowned.

I've just updated to 78.3.1 (64-bit) and now my LC_TIME locale settings are no longer recognised, again. Comment #144 did work again for the last year up until now

Jeff, thank you for the patch! I'll continue the work and hopefully get this landed.

Try job here: https://treeherder.mozilla.org/#/jobs?repo=try&revision=30560324f1e554b8d94059a5ea07ad457f37b9c7

Now that we've removed a lot of the special cases we were supporting in the date
and time format selectors, we can call GetDateTimePattern once with the two
selectors instead of calling it twice and combining the results. This will
simplifies the code and will make it easier to handle overriding patterns using
prefs.

A side effect of this change is that on OS X, you get a slightly different result
if you ask for the long format date and time all at once than if you ask for the
two separately and then combine them. The test expectations have been updated
accordingly.

This allows for the removal of duplicated code between DateTimeFormat and
OSPreferenes.

Depends on D94431

This new method allows for the user to override the long and short date and
time patterns by prefs. If no overrides are set or it fails while processing
the prefs, it will fallback to the existing methods for determining the patterns.
Since the user may have only overriden one of date or time, it is necessary to
be able to look up the other separately and combine the results.

This adds a prefix callback that watches the new prefs and flushes the cache if
they change. Otherwise, the user would have to restart the browser to see the
results of changing a pref, and would make testing more difficult. Unregistering
this callback required changes to the destructor, which was previously defined
separately on each operating system. A new RemoveObservers method has been added
to handle OS specific cleanup.

Depends on D94432

Updated try run here: https://treeherder.mozilla.org/#/jobs?repo=try&selectedTaskRun=LGHwQwTZQi-_J0szU9zyFw.0&revision=5e9993c0c0f664fec150a1f3cd4f3f5dd7f26cd4 and windows fixed here: https://treeherder.mozilla.org/#/jobs?repo=try&revision=ad9c3b00064b764271ec7897cc9f862d3be9136f.

The connector patterns can change over time, we should instead check for the date time components
that we expect to be present.

We'll want to use UTF-8 when we switch to using ICU4x because Rust is all UTF-8. We
can switch the external facing APIs now, and update the internal implementations
later.

Depends on D94432

https://hg.mozilla.org/mozilla-central/rev/7774282861fe
https://hg.mozilla.org/mozilla-central/rev/4602056737ce
https://hg.mozilla.org/mozilla-central/rev/e1137eef3ae3
https://hg.mozilla.org/mozilla-central/rev/1a5d1964ff10
https://hg.mozilla.org/mozilla-central/rev/5a4ccc7684fb

Fantastic!

Can you advise if / when a nightly is out that includes the fixes so I can try it?

Thanks!

It looks like it is in today's nightly, from [1] it looks like it was built from https://hg.mozilla.org/mozilla-central/rev/83bf4fd3b1fbca9dcbe23de9d1a1503eed62569a, which according to [2] would include these changes.

Please test and let us know if you encounter any issues. Thanks!

[1] http://ftp.mozilla.org/pub/thunderbird/nightly/2020/10/2020-10-28-10-21-40-comm-central/thunderbird-84.0a1.en-US.linux-i686.txt
[2] https://hg.mozilla.org/mozilla-central/pushloghtml

Thank you :dminor for taking ownership of this and fixing this long standing issue!

@all, This is just the platform scaffolding for the custom pattern overrides. It should allow Thunderbird and/or Firefox to write front end UI in Preferences to override patterns. It may also be possible to extend WebExtensions to allow extensions to override those.

For now, if you want to test it, you need to understand patterns: https://unicode.org/reports/tr35/tr35-dates.html#Date_Format_Patterns

And then add some or all of the following settings with your patterns of choice:

• intl.date_time.pattern_override.time_(short | medium | long | full)
• intl.date_time.pattern_override.date_(short | medium | long | full)

(In reply to Zibi Braniecki [:zbraniecki][:gandalf] from comment #182)

Thank you :dminor for taking ownership of this and fixing this long standing issue!

@all, This is just the platform scaffolding for the custom pattern overrides. It should allow Thunderbird and/or Firefox to write front end UI in Preferences to override patterns. It may also be possible to extend WebExtensions to allow extensions to override those.

For now, if you want to test it, you need to understand patterns: https://unicode.org/reports/tr35/tr35-dates.html#Date_Format_Patterns

And then add some or all of the following settings with your patterns of choice:

• intl.date_time.pattern_override.time_(short | medium | long | full)
• intl.date_time.pattern_override.date_(short | medium | long | full)

So good to see progress on this long-standing annoyance! Yay!!!

I downloaded the nightly Thunderbird and updated the prefs.js file for it to include the following:

user_pref("intl.date_time.pattern_override.date_full", "yyyy-MM-dd HH:mm:ss Z (EEEE)");
user_pref("intl.date_time.pattern_override.date_long", "yyyy-MM-dd HH:mm:ss Z");
user_pref("intl.date_time.pattern_override.date_medium", "yyyy-MM-dd HH:mm:ss");
user_pref("intl.date_time.pattern_override.date_short", "yyyy-MM-dd");
user_pref("intl.date_time.pattern_override.time_full", "yyyy-MM-dd HH:mm:ss Z (EEEE)");
user_pref("intl.date_time.pattern_override.time_long", "yyyy-MM-dd HH:mm:ss Z");
user_pref("intl.date_time.pattern_override.time_medium", "HH:mm:ss");
user_pref("intl.date_time.pattern_override.time_short", "HH:mm");

I wasn't sure where each of the various pattern overrides were applied, so this is just a guess (a big unknown for me is when the date override is used versus the time override).

This gets very close to an ISO 8601 format, so is a nice improvement. Unfortunately the date display in both the list of mails and in the mail display itself is still not completely correct, as for some reason it places a comma between the date and time, like this:

2014-12-10, 19:04

I guess that it's using something like "short date", "short time" which is possibly reasonable but what I really want is them concatenated without the comma in between. A slightly adjusted version of the preferences is even closer:

user_pref("intl.date_time.pattern_override.date_short", "yyyy-MM-dd HH:mm:ss");
user_pref("intl.date_time.pattern_override.time_short", " ");

(Note that there has to be a space in the time_short override, otherwise it is ignored and the default time formatting is used.)
This results in a time/date format with a comma at the end, like this:

2014-12-11 18:31:26,

This is probably good enough, but being able to get rid of that comma would complete the ISO 8601 ascension. :-D

Thank you for testing this :)

The problem is that we apply the overrides for date and time separate and then combine them by looking up the locale specific "connector" which in your case appears to be ,. I think the solution would be to add another pref which allows the user to override the connector as well.

I had deliberately ignored empty values for the overrides, so that the prefs could exist but not have an effect. I hadn't thought about the use case of putting the entire pattern in a single override, but if I implement the connector override, this shouldn't be necessary.

I'll check with Zibi, but I don't see any problems with adding an override for connectors.

I filed a follow up in Bug 1674212 to add an override pref for the connector pattern.

First, many many thanks for doing this! One question:

The date in the message-list pane for messages between 1-7 days old has
(for me) the format: EEE HH:mm eg: Thu 14:03
Will I be able to customize this format? I don't see it in any of
the example preferences.

To continue my previous comment, #186:
BTW, what I would like for myself is EEE-d HH:mm . I admit this is
just a personal preference, but this brings up another point:

I would like to say how much I love the approach of just letting the user
design the format for themself, avoiding all the endless decision-making
about the exact ISO-like variations to offer.
This solution strikes me as 100% kludge-free! THANKS!

(In reply to Dan Minor [:dminor] from comment #185)

I filed a follow up in Bug 1674212 to add an override pref for the connector pattern.

Great Dan, thank you very much. When a build arrives with that additional setting I'll give that a test.

I'm looking forward to having these date formatting options available! :-D

Good to see, this is hopefully solved by giving users a possibility to configure custom date/time patterns (instead of an endless debate how to name the date/time format everybody wants:).

Unfortunately for us poor Debian users we now are stuck on TB 78 ESR:

• This means I cannot benefit form the custom date/time patterns solution.

• It also means the "LC_ALL=root.UTF-8 thunderbird" workaround I use no longer appears to function. (Details of this workaround are described in various comments to this bug (comment 58+59, 74, 94, 113, 130, 138, 144, 151) and I also have my exact details described in bug 1502659 comment 2.)

Is there any solution/workaround suitable for TB 78 ESR?

(In reply to Bakhelit from comment #189)

Unfortunately for us poor Debian users we now are stuck on TB 78 ESR:

There's no reason Debian users can't download the newest TB release from thunderbird.net, unpack it locally, put it at the front of their search path, and use it instead of the Thunderbird that ships with Debian.

(In reply to Jonathan Kamens from comment #190)

There's no reason Debian users can't download the newest TB release from thunderbird.net, unpack it locally, put it at the front of their search path, and use it instead of the Thunderbird that ships with Debian.

Well, except that distributions tend to patch packages to fix various compatibility issues and to streamline integration with the rest of the OS. (openSUSE, which I use, applies 26 such patches to the current stable release of Thunderbird. I have no idea what Debian does.) But using the official release is worth a try, provided you first back up your profile. Any change/loss in functionality you may encounter may well be easier to live with than the date formatting issue.

I'm sorry, I could have been clearer. While you're correct that distributions do tend to include minor patches in their builds of applications like Thunderbird, I have personally used the version of Thunderbird distributed by the Thunderbird team on Debian-based systems on many occasions and I have not encountered any significant compatibility issues.

(In reply to Jonathan Kamens from comment #192)

I'm sorry, I could have been clearer. While you're correct that distributions do tend to include minor patches in their builds of applications like Thunderbird, I have personally used the version of Thunderbird distributed by the Thunderbird team on Debian-based systems on many occasions and I have not encountered any significant compatibility issues.

I prefer to have all software on my systems installed from Debian repositories, because it makes it possible to handle all software installations, removals and especially updates using one management system which greatly simplifies its configuration and scheduling of software updates.

My current systems for example handle all updates automatically without any user intervention and usually without any need to change configs of installed applications during the whole Debian stable release lifetime. In fact with the exceptions of Firefox and Thunderbird which in recent years often totally disrupt my configs with each ESR update, I usually get away with updating my applications configs only when switching to new Debian stable release. So, thanks for doing ESR, without it I would probably need to switch to some other browser and email client, because dealing with disrupted configs each time Mozilla's rapid release machine forces new non-ESR version on us would be unbearable for me.

Thus, if there is no chance of fixing TB 78 ESR I will have to accept it and wait for next TB ESR in Debian repositories. Still, I would like to know why the "root" locale workaround does not work any more? Did Unicode CLDR database removed the "root" fallback? Or did Mozilla change the locale handling again to for example force the default "en-US" instead of "root" locale?

(In reply to Bakhelit from comment #193)

I prefer to have all software on my systems installed from Debian repositories, because it makes it possible to handle all software installations, removals and especially updates using one management system which greatly simplifies its configuration and scheduling of software updates.

That is certainly a reasonable preference, and I strive to operate the same way myself.

However, that's not the same as "Unfortunately for us poor Debian users we now are stuck on TB 78 ESR." You're not. There is a workaround for that, which you can avail yourself of should you choose to do so. It's your choice not to, a perfectly reasonable one, but I don't think it's reasonable to portray that here as a trap with no way out.

It's just a fact of life that it's simply impossible for Thunderbird's release and bug-fix schedules to always align in the most optimal way with Debian's (or any other OS's).

The maintainers of Thunderbird put a lot of effort into putting out a lot of binary releases for a lot of platforms. I don't think it's particularly polite or generous to imply that those don't exist simply because you personally prefer not to stray from the packages distributed by your OS maintainer.

Sorry, I could wrote something less dramatic instead of "Unfortunately for us poor Debian users we now are stuck on TB 78 ESR." - at that time I could not see that it could be seen as ungrateful, impolite or whatever.

I was interested in finding out if there is a chance to consider using the fix from this bug also for TB 78 ESR. If it is not technically possible or is too much work, I was hoping to at least learn how to change/update my workaround for TB 78 ESR or if it is even possible at all.

I certainly do not want to imply no alternative options exist or that devs and maintainers of TB or FF do their job badly - quite the opposite I am sure they try do their best. I am also aware that even if they did ther job badly, users have no "moral" right to complain since many devs and maintainers do their work for free. So, I am of course very greatful for the work all TB and FF devs and maintainers do - that is why I at least try to help by reporting bugs/problems I find when configuring my TB and FF. I hope this explains my views better:).

For people wanting to test this awesome new feature, it wasn't immediately clear how to do so on Ubuntu given that thunderbird-trunk is no longer in ppa:ubuntu-mozilla-daily/ppa. Downloading the latest binary from https://archive.mozilla.org/pub/thunderbird/nightly/latest-comm-central/ (I downloaded 89.0a1) and adding the lines per https://bugzilla.mozilla.org/show_bug.cgi?id=1426907#c183 to the active profiles prefs.js file does the trick. Date formats are lovely:

2021-04-05, 20:02
00:17

I am not following how to set the connector override to eliminate the comma. That would be a readability improvement. This is an awesome usability enhancement.

The message pane seems to only support date_short, time_short and time_short, which is fine by me, this is my preference.

Calendar events seem to take the Date Text Format assigned, but despite adding the 8 user_pref definitions for time/date short/medium/long/full (and seeing them in config editor), only short and long are available (but "long" is actually the format specified as "full", including the day of week). This is also fine given the formatting overrides.

I've configured mine like:

user_pref("intl.date_time.pattern_override.date_full", "yyyy-MM-dd HH:mm v (EEEEEE)");
user_pref("intl.date_time.pattern_override.date_long", "yyyy-MM-dd HH:mm v");
user_pref("intl.date_time.pattern_override.date_medium", "yyyy-MM-dd HH:mm");
user_pref("intl.date_time.pattern_override.date_short", "yyyy-MM-dd");
user_pref("intl.date_time.pattern_override.time_full", "yyyy-MM-dd HH:mm v (EEEEEE)");
user_pref("intl.date_time.pattern_override.time_long", "yyyy-MM-dd HH:mm v");
user_pref("intl.date_time.pattern_override.time_medium", "HH:mm");
user_pref("intl.date_time.pattern_override.time_short", "HH:mm");

The patterns are parsed as expected - YAY!!

One quirk is that upcoming events in the today pane duplicate the time - as if the display is date_full time_short rather than just the date format ("long/full") specified in the preferences.

-David

Why is the LC_TIME=C.UTF-8 workaround not working anymore to display the international date format YYYY-MM-DD when replying to emails? I am using the deb package Thunderbird 78.7.1 (64-bit) from Lubuntu 20.04.
I saw other workarounds but they are only compatible with the beta / nightly version and according to https://www.thunderbird.net/en-US/organizations/, the stable version of Thunderbird is the ESR (currently version 78) unlike Firefox. I would not risk using the beta or nightly version for writing professional emails, there was already a terrible privacy issue with the stable version of Thunderbird in the past (https://bugzilla.mozilla.org/show_bug.cgi?id=1201782#c29), so I would recommend only using the beta or nightly version for testing without private content.

Please note that the connector override introduced in bug 1674212 in pref intl.date_time.pattern_override.date_time_short was renamed to intl.date_time.pattern_override.connector_short in bug 1706318 for consistency. There are also medium, long and full connectors, but they are currently not used.

Let's expose the awesome work which has happened here for the annals of history and get the record right!

Starting out from a rather peripheral change of behaviour of the output format when using LC_TIME=en_DK.utf8 (was that even a bug?), this bug has matured into a full-fledged RFE which has implemented much-requested pref overrides for date and time formats:

• intl.date_time.pattern_override.date_*
• intl.date_time.pattern_override.time_*
  Substitute * with any of short/medium/long/full. The short connector is used for all formats:
• intl.date_time.pattern_override.connector_short (implemented in bug 1674212, renamed in bug 1706318).

Patches landed on mozilla-central so this really belongs to Core: Internationalization, of which Firefox and Thunderbird are consumers.

I would like to thank everyone involved on this bug for implementing these awesome date and time formatting prefs, especially Dan M., Zibi, Jörg K., Jeff H., and Niklas H.! Great job! 👍🏻👍🏻👍🏻

If comment #199 was meant to be a summary, it's missing the information from comment #198:

• intl.date_time.pattern_override.date_*
• intl.date_time.pattern_override.time_*
  substitute * with any of short/medium/long/full. The short connector is used for all formats:
• intl.date_time.pattern_override.connector_short

(In reply to José M. Muñoz from comment #201)

If comment #199 was meant to be a summary, it's missing the information from comment #198:

Thank you! I've updated my comment accordingly. intl.date_time.pattern_override.connector_short was not implemented here in bug 1426907, but it's good to have the full summary here.

(In reply to Thomas D. (:thomas8) from comment #200)

I would like to thank everyone involved on this bug for implementing these awesome date and time formatting prefs, especially Dan M., Zibi, Jörg K., Jeff H., and Niklas H.! Great job! 👍🏻👍🏻👍🏻

Thank you! 😀

I'm somewhat mystified by much of the discourse within this bug, but the end result looks pretty understandable. I am looking forward to when this ends up in my distribution's Thunderbird package.

Thank you, very much, as well.

-Corey

(In reply to gessel@blackrosetech.com from comment #196)

This is an awesome usability enhancement.

I've configured mine like:

user_pref("intl.date_time.pattern_override.date_full", "yyyy-MM-dd HH:mm v (EEEEEE)");
user_pref("intl.date_time.pattern_override.date_long", "yyyy-MM-dd HH:mm v");
user_pref("intl.date_time.pattern_override.date_medium", "yyyy-MM-dd HH:mm");

I think that's not how you're supposed to use it. All the pattern_override.date_* prefs are only supposed to have date placeholders, not also time placeholders. Adding both date and time into the date pattern override looks like a recipe for UI failures imo. Someone correct me if I'm wrong.

user_pref("intl.date_time.pattern_override.date_short", "yyyy-MM-dd");
user_pref("intl.date_time.pattern_override.time_full", "yyyy-MM-dd HH:mm v (EEEEEE)");
user_pref("intl.date_time.pattern_override.time_long", "yyyy-MM-dd HH:mm v");

Same here - pattern_override.time_* should have times only, not dates also.

user_pref("intl.date_time.pattern_override.time_medium", "HH:mm");
user_pref("intl.date_time.pattern_override.time_short", "HH:mm");

The patterns are parsed as expected - YAY!!

Given your setup, that's quite surprising... probably just using the your short formats, which happen to be correct.

One quirk is that upcoming events in the today pane duplicate the time - as if the display is date_full time_short rather than just the date format ("long/full") specified in the preferences.

So then the duplicate time should be the result of your wrong setup, as explained above.
HTH.

(In reply to gessel@blackrosetech.com from comment #196)

For people wanting to test this awesome new feature, it wasn't immediately clear how to do so on Ubuntu given that thunderbird-trunk is no longer in ppa:ubuntu-mozilla-daily/ppa. Downloading the latest binary from https://archive.mozilla.org/pub/thunderbird/nightly/latest-comm-central/ (I downloaded 89.0a1) and adding the lines per https://bugzilla.mozilla.org/show_bug.cgi?id=1426907#c183 to the active profiles prefs.js file does the trick.

Thanks David for providing this helpful first explanation.

The feature is currently available in Thunderbird Beta 90.0b2 and Daily 91.0a1, both available for download from Thunderbird homepage.

How to create date and time format override prefs using Thunderbird's Config Editor

It is possible, but not required to edit prefs.js directly - you can just use Thunderbird's inbuilt config editor and create these prefs.
≡ > Preferences > General > [Config Editor...] button at the very bottom.

Type the full name of the pref which you want to create into the search box:
intl.date_time.pattern_override.date_short
As the pref does not exist, it will show up in the results list as a new pref which you can create by first selecting (o) String and then using the the [+] button (marked purple in the screenshot). It will then ask you for a value, and you can enter yyyy-MM-dd (you can always edit the value again later). Then do the same by analogy for intl.date_time.pattern_override.time_short.

Date formats are lovely:

2021-04-05, 20:02
00:17

I am not following how to set the connector override to eliminate the comma. That would be a readability improvement.

How to change the date/time connector (e.g. from comma to space)

As seen in the screenshot: The connector must have date and time placeholders in curly brackets.

• Date: {1}
• Time: {0}
• Strings must be single-quoted to avoid parsing, but some simple characters like space () or comma (,) work without quoting.
• Apparently the format follows https://unicode.org/reports/tr35/tr35-dates.html#Date_Time_Combination_Examples

intl.date_time.pattern_override.connector_short = {1} {0} (single space between placeholders)
Result for short date and short time combo (see screenshot):
2021-06-24 21:00

Regular ASCII characters which you want to display must be single-quoted (otherwise the date-time display may get truncated):

intl.date_time.pattern_override.connector_short = {1}'T'{0}
Result for short date and short time combo:
2021-06-24T21:00

Hope that helps!

Thomas

Apparently the pattern format is documented here: https://unicode.org/reports/tr35/tr35-dates.html#Date_Format_Patterns

We still need documentation for the overrides. Guessing from the names, I'd assume that intl.date_time.pattern_override.time_* is time only (as in clock, e.g. 23:59) and intl.date_time.pattern_override.date_ is supposed to be date (as in "2021-06-28") without a time and intl.date_time.pattern_override.connector_short defines how those should be combined. Default seems to be {1}, {0} meaning that you get date followed by comma, space and time. The actual implementation seems to require both {0} and {1} so you cannot customize that too much: https://hg.mozilla.org/mozilla-central/file/tip/intl/locale/OSPreferences.cpp

The expected mapping for e.g. E vs EE Vs EEE vs EEEE vs EEEEE vs EEEEEE should be defined somewhere for Thunderbird.

Also, how to combine short date with long time for the message reply header?

This link seems to be the maintained ICU documentation page for CLDR date symbols:
https://unicode-org.github.io/icu/userguide/format_parse/datetime/#date-field-symbol-table

H(In reply to Thomas D. (:thomas8) from comment #206)
Fixing my preferences so I think they comply with comment 206, (thanks Thomas, it helped a lot!)

intl.date_time.pattern_override.connector_short	{1} {0}	
intl.date_time.pattern_override.date_full	yyyy-MM-dd (EEEEEE)
intl.date_time.pattern_override.date_long	yyyy-MM-dd	
intl.date_time.pattern_override.date_medium	yy-MM-dd	
intl.date_time.pattern_override.date_short	MM-dd	
intl.date_time.pattern_override.time_full	yyyy-MM-dd HH:mm Z z
intl.date_time.pattern_override.time_long	yyyy-MM-dd HH:mm z
intl.date_time.pattern_override.time_medium	HH:mm Z
intl.date_time.pattern_override.time_short	HH:mm

I notice that only the short form of time seems to be used. As in comment 207 above, there are some limits to using these new options in the calendar. The only mailbox pane guidance I can find is this http://kb.mozillazine.org/Date_display_format - which also doesn't reference long/short time. With the above time.pattern_override settings and experimenting with the preference mail.ui.display.dateformat.today values I get the following results:

It'd be awesome to add the following to make full use of this lovely feature:

Sadly you are correct, mail.ui.display.dateformat.today values 3 and 4 don't work any more :-(

They had fallen under the knife for a short while in TB 53, but got reimplemented in bug 1329841. Internally they related to kDateFormatYearMonth and kDateFormatWeekday. Now they've been removed again in bug 1669573.

... removed in bug 1669573 and bug 1671532.

Testing the remaining values of mail.ui.display.dateformat.today
0 gives S13:04 with intl.date_time.pattern_override.time_short set to 'S'HH:mm
1 gives L2021-06028, S13:04 with intl.date_time.pattern_override.date_long set to 'L'yyyy-MM-dd
2 gives S21-06028, S13:04 with intl.date_time.pattern_override.date_short set to 'S'yy-MM-dd
That confirms your observations. BTW, why do you have a date pattern yyyy-MM-dd in two of your time overrides?

And finally there is bug 1672548 which is about restoring at least the weekday display for mail.ui.display.dateformat.* value 4.

DOH!
Jose, the answer is just incomplete clean up. I was putting random parameters in to see where they showed up and never finished cleaning up the mistake @thomas8 pointed out.

Thanks for the link to bug 1672548. I'd argue that with the new override pattern features which really cover pretty much any concievable use case, the way to leverage that configuration power in display formats is to enable calling them. As indicated in my proposed preferences modification, calling time and date formats individually provides a nice combinatoric feature effect from the explicitly individual preference configuration for {date} and {time}, but my needs would be more than sufficiently suited if mail.ui.display.dateformat.X took the form:

Likewise, modifying the preferences->calendar->Date Text Format: drop down selector to likewise enable selection of short/medium/long/full would seem consistent with the purpose of the new preferences.

one question: intl.date_time.pattern_override.connector_short implies intl.date_time.pattern_override.connector_(short|medium|long|full) - is this true?

You are missing the original option 4 that bug 1672548 tries to restore: "Fri, 10:55". Yes, the short connector pattern is always used, see 1706318 comment #3.

"Let's hope that no one asks for the long version." :-)

I admit I'm coming to this topic from the long thread about requesting treating ISO 8061 as a special case outside of ICU localization and through that lens I can't help but see creating exception formats for mail.ui.display.dateformat.X in a similar light. Is there a reason why my proposal to map the mail.ui.display.dateformat.X interpretations to the intl.date_time.pattern_override.X_X pattern definitions wouldn't achieve your goal? If you wanted value "4" to correspond to "Fri, 10:55" it could be achieved with:

intl.date_time.pattern_override.connector_short     {1},{0}
intl.date_time.pattern_override.time_full           HH:mm
intl.date_time.pattern_override.date_full           ccc

I apologize if I'm letting my excitement for arbitrary ICU date field symbol construction get in the way.

Oops, yes, if want to misuse the "full" to be a weekday only and then drive the mail.ui.display.dateformat.thisweek to use this 'fake' "full", then it can be done. BTW, should it be ccc or E?

I have documented this feature for Thunderbird:

• https://support.mozilla.org/kb/customize-date-time-formats-thunderbird
• https://enterprise.thunderbird.net/manage-updates-policies-and-customization/thunderbird-preferences-enterprise/customize-thunderbirds-date-and-time-formats

Also encountered SUMO Bug 1723190 on the way - cunning!