I can confirm this on 60.6.1: email without a Date: header are shown with date of IMAP sync (the date of the file on the local thunderbird disk). This morning this produced a big family quarrel on an email sent by our alarm/intusion system.
Please fix it, and make the last Received header date as the displayed date, and not the file download date.

In my case this bug seems to be fixed by setting "mailnews.customDBHeaders" to "Received" in config editor. Some other users also confirm that this fixed their problems (see http://blog.dmitryleskov.com/small-hacks/putting-the-received-column-in-thunderbird-to-work/). Is it a possible solution?

I experienced this bug and can confirm it is still happening as of the current latest Thunderbird 78. I imported a large imap account to thunderbird with thousands of emails, and a handful of them apparently had improperly formatted headers so the dates of those emails were shown as being received during the import process, instead of several years ago. The blog post mentioned by osdm fixed it for me, and now all of those emails are sorted correctly during the date which they were actually received.

I have this problems with email sent by my alarm/intusion system.
Please fix it, and make the last Received header date as the displayed date, and not the file download date.

(In reply to Matthew Jurgens from comment #22)

This is still an issue with TB 64 Beta 2 using an IMAP server with maildir.

The only date header that these mails contain is
Delivery-date: Mon, 19 Mar 2018 09:36:43 +1100

However, the date is shown as the date that the mail was downloaded from the
server (presumably the file modification date or something). I just added
the IMAP account to a new computer and hence the mails were downloaded and
the date shown in the GUI as the download date/time.

See the attachment: https://bugzilla.mozilla.org/attachment.cgi?id=9025855
for the GUI view.

So the question is either we may want to add "Deliver-date:", which is not quite standard IMHO,
See Bug 612990
or
do something with the latest "Received:" header lines.
Actually, Bug 612990 comment the first Received header we see from the start (the last added Received: )
usually is very close to Delivery-date:.
So we might simply do that.

One thing I noticed lately during the fix for a different bug 1699968
is that
TB may not properly handles the timestamp when mDatabase is null.
It does not call a function to set time stamp or something to that effect.
So this observation may not affect a particular message in question, but that may need be kept in mind.
(This is actually a reminder to myself.)

(In reply to bugzilla from comment #28)

I have this problems with email sent by my alarm/intusion system.
Please fix it, and make the last Received header date as the displayed date, and not the file download date.

So there is a sizeable different time between Recevied header date and the file download date?

Of course, the problem is with the sender (maybe automated web CGI/JS or whatever) which does not bother to add "Date:" line.
But in communication, the golden rule is try to be strict about the protocol conformance with what we send out,
but a bit lenient about what we receive.

All you need is to enable the download of the Received header (Bug 402594).
You can do so through adding 'received' to the 'mailnews.customDBHeaders' pref (Bug 1361643 bug comment #22).

Or use Menu: View -> Messages -> Customize... which will set 'mailnews.customHeaders' pref which will also effect customDBHeaders.
https://searchfox.org/comm-central/rev/3a01c489963cb3bc423bb3b78e9cbecbec6b99fe/mailnews/local/src/nsParseMailbox.cpp#475

(In reply to ISHIKAWA, Chiaki from comment #29)

So the question is either we may want to add "Deliver-date:", which is not quite standard IMHO,
See Bug 612990

Regarding standard or non-standard, I have no concerns. The question is rather whether we should edit the content of the e-mail at all. Or would it be more appropriate to introduce an X-Mozilla-DeliveryDate header?

In any case, this would only be a solution for POP3 or local folders. Uploading the modified mail back to an IMAP account is probably out of the question.

I've always understood that:
'Date' is the date the composer created the email. That email may have been created at night, but not actually sent until the morning.
That date may appear to be a different day if significant timezone is involved.

'Received' date is the date the receiving server for mail account actually received the email. In headers that would be the one at the top.

The downloaded date is different again because email may have been on server for days, but computer with Thunderbird installed was not switched on etc. Although in most cases, emails are received by server and downloaded soon thereafter.

The dates displayed can be adjusted depending upon the date/time/timezone set up on computer which is running Thunderbird.
But you would assume all computers do have correct date/time setup.

Example of date:
Bugzilla email says comment changed in bug: at 2022-06-16 18:36:21 PDT

'Date' in received email says: Fri, 17 Jun 2022 01:36:22 +0000
'Received' on server date: Fri, 17 Jun 2022 02:36:25 +0100

But the info at top of 'Source View' says : From - Fri Jun 17 11:12:30 2022 and that was the time I started Thunderbird and downloaded the email. Email various dates - clearly adjusting for timezones as I'm in the UK.

I have preference: mailnews.customDBHeaders Received
It works perfectly on both pop and imap accounts.

In my case, I had inadvertantly set this up whilst doing some filtering some time ago - I had no idea this had fixed the bug, so was perplexed when others had the problem.

This is a request for consideration to perform a quick fix on this bug, so it can be marked as resolved.
Perhaps we could resolve this bug if this preference was set by default: mailnews.customDBHeaders Received
Can this be done for the next update as it is not too complicated?
Thoughts on whether this would be a reasonable action?

(In reply to Alfred Peters from comment #35)

(In reply to ISHIKAWA, Chiaki from comment #29)

So the question is either we may want to add "Deliver-date:", which is not quite standard IMHO,
See Bug 612990

Regarding standard or non-standard, I have no concerns. The question is rather whether we should edit the content of the e-mail at all. Or would it be more appropriate to introduce an X-Mozilla-DeliveryDate header?

In any case, this would only be a solution for POP3 or local folders. Uploading the modified mail back to an IMAP account is probably out of the question.

Right. I was thinking of POP3 or local mail folder situation more or less.
I need someone with heavy IMAP usage experience to comment on the matter.

(In reply to Anje from comment #38)

I've always understood that:
'Date' is the date the composer created the email. That email may have been created at night, but not actually sent until the morning.
That date may appear to be a different day if significant timezone is involved.

Correct. The Date header. But some mail clients don't set this header.

'Received' date is the date the receiving server for mail account actually received the email. In headers that would be the one at the top.

The "Received Header" is a receipt stamp generated by the respective server through which the message was transported in order to be able to trace the transport route. The date field is only a side information and IIRC even optional.

The downloaded date is different again because email may have been on server for days, but computer with Thunderbird installed was not switched on etc. Although in most cases, emails are received by server and downloaded soon thereafter.

Correct. We store this date in our folder database. You can display it in the folder pane by showing the 'Received' column.
Unfortunately, this information is lost when you perform a "Repair Folder".
Therefore, it would be helpful to additionally store this information in the header of the mail itself:

(In reply to ISHIKAWA, Chiaki from comment #39)

(In reply to Alfred Peters from comment #35)

Regarding standard or non-standard, I have no concerns. The question is rather whether we should edit the content of the e-mail at all. Or would it be more appropriate to introduce an X-Mozilla-DeliveryDate header?
Right. I was thinking of POP3 or local mail folder situation more or less.

I agree with you.

(In reply to Anje from comment #38)

I have preference: mailnews.customDBHeaders Received
It works perfectly on both pop and imap accounts.

IMAP is a different case - handled by Bug 402594.

What we are talking about here is the Date column. As source we should then use the earliest available date. If available of course also the Received header. This should be available when receiving via POP3.

I agree with Alfred Peters: in this thread we are talking about on what is shown on the Date column, and how this column is ordered.
An e-mail without a Date: header currently shows the date of the 1st download from the IMAP server in the Date column, so you can never find that email in the correct chronological order.
Thunderbird should guess the value to show on the Date column using the Received: headers, when Date: header is missing.
No modifications to the e-mail are needed.

(In reply to Alfred Peters from comment #40)

The "Received Header" is a receipt stamp generated by the respective server through which the message was transported in order to be able to trace the transport route. The date field is only a side information and IIRC even optional.

The downloaded date is different again because email may have been on server for days, but computer with Thunderbird installed was not switched on etc. Although in most cases, emails are received by server and downloaded soon thereafter.

Correct. We store this date in our folder database. You can display it in the folder pane by showing the 'Received' column.

I have just tested a email without Date header and am surprised by the result.
With POP3, the Received column shows me the date from the Received header and the Date column shows me the time of receipt in the TB. This should be the other way around in any case.

It should be:
Date column -> sent date -> earliest date
Received column -> receive date -> latest date

Has it always been like this? Or could this be an regression? Maybe by Bug 1707548?

(In reply to Alfred Peters from comment #43)

(In reply to Alfred Peters from comment #40)

The "Received Header" is a receipt stamp generated by the respective server through which the message was transported in order to be able to trace the transport route. The date field is only a side information and IIRC even optional.

The downloaded date is different again because email may have been on server for days, but computer with Thunderbird installed was not switched on etc. Although in most cases, emails are received by server and downloaded soon thereafter.

Correct. We store this date in our folder database. You can display it in the folder pane by showing the 'Received' column.

I have just tested a email without Date header and am surprised by the result.
With POP3, the Received column shows me the date from the Received header and the Date column shows me the time of receipt in the TB. This should be the other way around in any case.

Which version of TB did you use?

I have checked using the latest C-C TB locally compiled.
I send an e-mail without Date header (that is what I thought, but read on)
the Received and Date fields showed the time it was sent.

Well, I thought /usr/lib/sendmail under linux once allowed me to send an e-mail without Date: header line attached from
the sender, but it seems that it might have added Date: header automatically (or did the dovecot server that received the e-mail added it or did TB add that?).
I found that the header lines in the e-mail contained Date: header line.
E.g.:

From - Mon Jun 20 13:56:24 2022
X-Account-Key: account1
X-UIDL: 000157b2533eeb2c
X-Mozilla-Status: 0001
X-Mozilla-Status2: 00000000
X-Mozilla-Keys:                                                                                 
Return-Path: <ishikawa@ip030.example.localdomain>
X-Original-To: ishikawa
Delivered-To: ishikawa@ip030.example.localdomain
Received: by ip030.example.localdomain (Postfix, from userid 1000)
	id D0CDF271; Mon, 20 Jun 2022 13:38:42 +0900 (JST)
From: ishikawa@ip030.example.localdomain
Subject: 13:38
Message-Id: <20220620043842.D0CDF271@ip030.example.localdomain>
Date: Mon, 20 Jun 2022 13:38:24 +0900 (JST)  <--- This obviously was the time it was sent.

test without Date mail

I read the mail at about 13:57.

OK, I further checkedd. So actually, /usr/lib/sendmail on my Debian GNU/linux is one of the postfix suite programs in disguise.

ishikawa@ip030:/NREF-COMM-CENTRAL/mozilla/comm$ strings /usr/lib/sendmail | grep -i post
libpostfix-global.so
libpostfix-util.so
/usr/lib/postfix
the Postfix sendmail command has set-uid root file permissions
the Postfix sendmail command must be installed without set-uid root file permissions
option -V is deprecated with Postfix 2.3; specify -XV instead
option %s is deprecated with Postfix 2.3; specify -X%s instead
/etc/postfix
postqueue
%s/postdrop -r
postalias
/usr/lib/debug/.dwz/x86_64-linux-gnu/postfix.debug
ishikawa@ip030:/NREF-COMM-CENTRAL/mozilla/comm$ 

And postfix automatically add Date: header line if the mail message is missing it if configuration file says so.
From: https://www.postfix.org/postconf.5.html

always_add_missing_headers (default: no)
    Always add (Resent-) From:, To:, Date: or Message-ID: headers when not present. Postfix 2.6 and later add these headers only when clients match the local_header_rewrite_clients parameter setting. Earlier Postfix versions always add these headers; this may break DKIM signatures that cover non-existent headers. The undisclosed_recipients_header parameter setting determines whether a To: header will be added.

BUT I don't see |always_add_missing_headers| in main.cf for postfix. But Debian is known to tweak configuration files of programs for maintenance purposes and I may have missed it.

Now as for , my pop3 server that I use for testing is part of dovecot suite. When I send the e-mail with

/usr/lib/sendmail ishikawa
From: ishikawa
Subject: test header line

    ...
.

The mail is delivered to local dovecot mail server and that could be when missing Date: header line may have been added.

So many environmental setup factors to get right.

It should be:
Date column -> sent date -> earliest date
Received column -> receive date -> latest date

I agree. At least Received should be the latest. But Received is the received time the last mail server from which one reads
received the e-mail. Not the time you read the e-mail from the mail host.

So in my case, since the delivery to the final mail server is about 20 seconds I sent the mail. So Received: shows very similar time to Date.

Now, I have to figure out how to send an e-mail without Date: reliably.
(Maybe I replace postfix-in-disguise sendmail with a real sendmail.)

Has it always been like this? Or could this be an regression? Maybe by Bug 1707548?

I thought it was OK before. I have to find an environment to test this correctly.

(In reply to Anje from comment #38)

Perhaps we could resolve this bug if this preference was set by default: mailnews.customDBHeaders Received
Can this be done for the next update as it is not too complicated?

It could be considered in general. It would have known and unknown performance - and perhaps other - implications though. So it would need time and analysis.

(In reply to Alfred Peters from comment #43)

I have just tested a email without Date header and am surprised by the result.
With POP3, the Received column shows me the date from the Received header and the Date column shows me the time of receipt in the TB. This should be the other way around in any case.

It should be:
Date column -> sent date -> earliest date
Received column -> receive date -> latest date

Has it always been like this? Or could this be an regression? Maybe by Bug 1707548?

pop3-js didn't do any headers processing, parsing and saving are delegated to nsIPop3Sink.

Perhaps it would be easier to understand if there was more clarity on what 'Received' actually means rather than leave it's contents be interpreted on an adhoc basis.
It has always been 'received on server' - so would it help clarify the meaning of what people see if that column header was renamed to 'Server Received'. So it's functionality and contents do not change and adding the 'Received' to the mailnews.customDBHeaders then works as well.

If you have the 'Received' aka 'Server Received' date working it solves getting the Server received date instead of a duplicate created date in the Date column. This is a quick fix for one part of the problem.

'Date' has always been the date the email was composed and not the date/time sent. Although in most cases it would appear to be about the same.
For example email composed at 16:41 hrs and use 'Send Later'. Wait until eg: 16:45 and then then send unsent messages.
The received incoming email will have 'Date' as 16:41 (obviously this was composed and sent same day) and not 16:45hrs.

But for email sent with no date then it seems Thunderbird applies alternative being 'Download Date'.
I see the options being:

1. Either do nothing with 'Date' as it seems entering the download date is a good alternative for those few emails with no date.
2. Leave 'Date' empty if no date and offer another new 'Download Date' column.

I think there is a good arguement for creating a new 'Download Date' column anyway as sometimes you want to know the difference in time between:
Date email was created.
Date email was received on server.
Date email was downloaded.

Knowing when someone composed email and prepared to send, when server received it and when you downloaded from server would be convenient.
Thoughts ?

Adding extra columns to fix this sorting problem is not an option for me: my users will never be able to understand the problem, come here and read this thread and then add the missing column by themselves. Thunderbird should not confuse the user by default, not "only if configured" with an extra column.

Filling the missing Date: column information with the download date is WRONG for two reasons:

• when configuring a new thunderbird, all e-mail missing Date: headers will be the last and more recent in the list. Even e-mails that are months old will appear as received now.
• it's misleading: a device sends an alert during the night, without Date: header. You open thunderbird at 11:00 AM. And then the alert appears to be sent at 11:00. This already happened to me and caused some problems in my family.

In my point of view, in the case of missing Date: header, thunderbird should choose to show the user the nearest date to the missing Date: header. And, if I'm not wrong, the nearest date to the e-mail composition is in the last Received: header.

(In reply to ISHIKAWA, Chiaki from comment #44)

(In reply to Alfred Peters from comment #43)

I have just tested a email without Date header and am surprised by the result.
With POP3, the Received column shows me the date from the Received header and the Date column shows me the time of receipt in the TB. This should be the other way around in any case.

Which version of TB did you use?

Tested with TB91.10 as well as current Daily.

The mail is delivered to local dovecot mail server and that could be when missing Date: header line may have been added.

The Date header is one of only two headers that are actually mandatory in an e-mail according to RFC 5322.

Therefore RFC 5321 also explicitly allows SMPT servers to add this header afterwards:
RFC 5321 - 6.4. Compensating for Irregularities
| The following changes to a message being processed MAY be applied
| when necessary by an originating SMTP server, or one used as the
| target of SMTP as an initial posting (message submission) protocol:
[...]
| o Addition of a date, time, or time zone when none appears

If a POP3/IMAP server makes up for the omission, no one will complain either. Unfortunately, there still seem to be enough clients/servers that don't do this. Otherwise we would not need this bug.

I agree with GiovanniP. All such tech details are not meaningful for us users. What is meaningful is a sensible behavior. When I install Thunderbird on a new device or when I do a repair for a folder, some old e-mails suddenly move to the most recent. Not only that surprises me, it also makes it hard to find an e-mail when you remember approximate date when it was received. For example, I may remember that an e-mail was about "ticket" (but it's not enough to find it, as search may give me hundreds of such e-mails) and that it was received in summer 2019. If such an e-mail is missing Date: header, then I won't find it, because I won't remember that I need to switch to Received (and I've read above that some other magic with customDBHeaders may be needed, I certainly won't remember about that when I reinstall TB).
A sensible behavior would be using something stable for a column that is displayed by default and used for sorted by default. For me it would be acceptable if you would use "Received" date by default, but that would be a breaking behavior for some other users, I don't know. So, the only "safe" solution is to substitute Received date instead of downloaded date when Date: header is missing (and do all other magic with customDBHeaders by default if needed).

I just examined Roundcube Webmail source code: it seems that by default it uses IMAP INTERNALDATE. And the result is good: e-mails are sorted according to the date they arrived on the IMAP server.
IMAP INTERNALDATE could be an alternative to the 1st Received: header.

(In reply to Anje from comment #47)

Perhaps it would be easier to understand if there was more clarity on what 'Received' actually means rather than leave it's contents be interpreted on an adhoc basis.

Agreed

It has always been 'received on server'

Okay, then I must have remembered that wrong.

• so would it help clarify the meaning of what people see if that column header was renamed to 'Server Received'. So it's functionality and contents do not change and adding the 'Received' to the mailnews.customDBHeaders then works as well.

Again: That is a solution for IMAP. Which is covered in Bug 402594.

For POP3 we already have the header. Its date is already in the Received column. If we don't have a better date, we should just copy it from there.

Actually, that's probably the plan - it just doesn't work:
https://searchfox.org/comm-central/rev/57645c6b44f6de29a414784105b8e2316e218428/mailnews/local/src/nsParseMailbox.cpp#1358

        // 'Received' should be as reliable an indicator of the receipt
        // date+time as possible, whilst always giving something *from
        // the message*.  It won't use PR_Now() under any circumstance.
        // Therefore, the fall-thru order for 'Received' is:
        // Received: -> Delivery-date: -> date
        // 'Date' uses:
        // date -> 'Received' -> PR_Now()

If you have the 'Received' aka 'Server Received' date working it solves getting the Server received date instead of a duplicate created date in the Date column. This is a quick fix for one part of the problem.

Agreed

'Date' has always been the date the email was composed and not the date/time sent. Although in most cases it would appear to be about the same.

JFTR: RFC 5322 - 3.6.1. The Origination Date Field
| The origination date specifies the date and time at which the creator
| of the message indicated that the message was complete and ready to
| enter the mail delivery system. For instance, this might be the time
| that a user pushes the "send" or "submit" button in an application
| program. In any case, it is specifically not intended to convey the
| time that the message is actually transported, but rather the time at
| which the human or other creator of the message has put the message
| into its final form, ready for transport. (For example, a portable
| computer user who is not connected to a network might queue a message
| for delivery. The origination date is intended to contain the date
| and time that the user queued the message, not the time when the user
| connected to the network to send the message.)

But for email sent with no date then it seems Thunderbird applies alternative being 'Download Date'.

Actually, only if no other date is available, this should be the last choice.

If I understood him correctly, Chiaki ISHIKAWA's suggestion is now to cache this date in a separate header when downloading for the first time so that it is not lost by a "repair folder".