SMTP 1000-character line limit violated by quote - causes invalid DKIM signatures on outbound mail.
Categories
(MailNews Core :: Networking: SMTP, defect)
Tracking
(Not tracked)
People
(Reporter: mozbz, Unassigned)
References
Details
Attachments
(5 files)
User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Firefox/91.0
Steps to reproduce:
When quoting a received email in plaintext where the original email had a line that hit the 1000-characther SMTP line-length limit (including the CRLF), Thunderbird will happily send a line that is longer than 1000 characters over the wire to the configured SMTP server, violating the RFC.
While this probably mostly shouldn't be an issue, in the case of Postfix + a DKIM-signing milter, this may cause the generated DKIM signature to be invalid as the MTA may truncate the line after the DKIM signature is generated [1].
In my specific case of a pretty standard Thunderbird + Postfix + opendkim setup, this does generate invalid DKIM signatures on outbound mail.
[1] https://marc.info/?l=postfix-users&m=163176643901328&w=2
Comment 1•3 years ago
|
||
Thanks for reporting, can you provide an eml file so that it's easier to try to reproduce this bug?
Updated•3 years ago
|
Attached two files - one is a test email received by Thunderbird, the second is the reply to said test email (by simply hitting the reply button in plaintext mode). The reply email is sent back to the server with an invalid line length (and you'll note it was sent to a DKIM validator, which confirmed the DKIM signature that it ended up with was invalid).
Updated•3 years ago
|
Updated•3 years ago
|
Comment 5•3 years ago
|
||
Thanks, can you please download a nightly from https://ftp.mozilla.org/pub/thunderbird/nightly/latest-comm-central-l10n/thunderbird-94.0a1.en-US.linux-x86_64.tar.bz2 and try to reproduce it? I think it's fixed in bug 1729437
Updated•3 years ago
|
Comment 6•3 years ago
|
||
Confirming based on reporter's expertise. This is actually pretty serious when it happens. Ping thinks it might be fixed by bug 1729437. Let's hear from Matt (reporter) if that's true...
Comment 7•3 years ago
|
||
(In reply to Thomas D. (:thomas8) from comment #6)
Confirming based on reporter's expertise. This is actually pretty serious when it happens. Ping thinks it might be fixed by bug 1729437. Let's hear from Matt (reporter) if that's true...
So, was it? Should this bug be made a duplicate?
Hi, sorry for the delay on this one.
Anyway, the immediate bug does appear to be fixed in 91.2.0, but there is a second related violation of the 1000 character limit which is still broken - I cannot attach a .eml which violates the 1000 character limit to a mail.
Comment 9•3 years ago
|
||
You're right, it's an old problem. Since bug 1255071, eml attachments are injected into the final message without any transformation. Seems gmail web does the same. I guess it's like this because not many people will attach "invalid" eml files. Even if they do, sending/receiving works in most cases.
Reporter | ||
Comment 10•3 years ago
|
||
Yes, today I discovered: (a) vger.kernel.org doesn't limit line lengths, then when trying to forward the .eml to the postmaster there, (b) Thunderbird doesn't handle that .eml correctly, and (c) iOS Mail also doesn't handle that .eml correctly. Seems this is kinda broken everywhere, but, hey, doesn't mean it isn't worth fixing :)
Comment 11•3 years ago
|
||
According to https://datatracker.ietf.org/doc/html/rfc2045#section-6.4
Certain Content-Transfer-Encoding values may only be used on certain
media types. In particular, it is EXPRESSLY FORBIDDEN to use any
encodings other than "7bit", "8bit", or "binary" with any composite
media type, i.e. one that recursively includes other Content-Type
fields. Currently the only composite media types are "multipart" and
"message".
So we can't encode it with base64. Inserting line breaks may work in some cases, but I'm not sure.
Reporter | ||
Comment 12•3 years ago
|
||
RFC 1341 seems to say this, however, implying the "correct" answer would be "binary".
The difference between "8bit" (or any other conceivable
bit-width token) and the "binary" token is that "binary"
does not require adherence to any limits on line length or
to the SMTP CRLF semantics, while the bit-width tokens do
require such adherence. If the body contains data in any
bit-width other than 7-bit, the appropriate bit-width
Content-Transfer-Encoding token must be used (e.g., "8bit"
for unencoded 8 bit wide data). If the body contains binary
data, the "binary" Content-Transfer-Encoding token must be
used.
Reporter | ||
Comment 13•3 years ago
|
||
(not that "binary" will solve the issue here, but maybe it will encourage BDAT usage, dunno if Thunderbird supports BDAT instead of DATA)
Comment 14•3 years ago
|
||
(In reply to Ping Chen (:rnons) from comment #11)
According to https://datatracker.ietf.org/doc/html/rfc2045#section-6.4
Certain Content-Transfer-Encoding values may only be used on certain
media types. In particular, it is EXPRESSLY FORBIDDEN to use any
encodings other than "7bit", "8bit", or "binary" with any composite
media type, i.e. one that recursively includes other Content-Type
fields. Currently the only composite media types are "multipart" and
"message".So we can't encode it with base64. Inserting line breaks may work in some cases, but I'm not sure.
I think you have misunderstood. The paragraph refers only to the parent structure of a multipart MIME message, because this could itself contain headers again.
The body of a single MIME part, which does not contain any other MIME structure, may very well be BASE64 or similar encoded.
IMHO this encoding may also be changed. If I remember correctly, this results from the SMTP RFC. There it is mentioned that transport servers are allowed to change the encoding if it is necessary e.g. for the 7-bit transmission.
Reporter | ||
Comment 15•3 years ago
|
||
I'm not sure if you're subscribed to mailop, but Brandon Long from Google seems to have agreed with Ping's understanding at https://list.mailop.org/private/mailop/2021-October/020231.html saying this about GMail:
Also, there is a tension in the rfc's here, because you aren't allowed to
encode message/rfc822, so you would have to edit the attached file.We used to base64 encode the message/rfc822 anyways, but we had enough
receivers not able to handle that, so we had to revert. We debated just
using message/global instead, but I think even fewer receivers understood
that.Of course, we also will happily use BDAT if the remote server supports it,
which doesn't have the line length limitation.
Comment 16•3 years ago
|
||
(In reply to Matt from comment #15)
I'm not sure if you're subscribed to mailop, but Brandon Long from Google seems to have agreed with Ping's understanding at https://list.mailop.org/private/mailop/2021-October/020231.html saying this about GMail:
No, so I do not know the context.
But again: It is not allowed to pack the complete message/rfc822 message in BASE64 encoding.
But it is allowed to change the body of the message.
I changed attachment 9241634 [details] as an example.
Reporter | ||
Comment 17•3 years ago
|
||
No, so I do not know the context.
There's not a lot of context you're missing, basically just this current discussion. I read his response as "we used to violate the RFC and encode the full attachment as base64, because doing otherwise is somewhat nonsensical, but receivers had problems with it, so GMail stopped doing that", which I figured was useful context for this discussion :).
Comment 18•3 years ago
|
||
This is not allowed!
Comment 19•3 years ago
|
||
Multipart message with changed body type - this is OK
Updated•3 years ago
|
Updated•3 years ago
|
Comment 20•3 years ago
|
||
(In reply to Alfred Peters from comment #14)
(In reply to Ping Chen (:rnons) from comment #11)
According to https://datatracker.ietf.org/doc/html/rfc2045#section-6.4
Certain Content-Transfer-Encoding values may only be used on certain
media types. In particular, it is EXPRESSLY FORBIDDEN to use any
encodings other than "7bit", "8bit", or "binary" with any composite
media type, i.e. one that recursively includes other Content-Type
fields. Currently the only composite media types are "multipart" and
"message".So we can't encode it with base64. Inserting line breaks may work in some cases, but I'm not sure.
I think you have misunderstood. The paragraph refers only to the parent structure of a multipart MIME message, because this could itself contain headers again.
The body of a single MIME part, which does not contain any other MIME structure, may very well be BASE64 or similar encoded.
IMHO this encoding may also be changed. If I remember correctly, this results from the SMTP RFC. There it is mentioned that transport servers are allowed to change the encoding if it is necessary e.g. for the 7-bit transmission.
I think there is no misunderstanding. The paragraph says for any MIME part, when Content-Type is multipart/* or message/*, the Content-Transfer-Encoding can only be 7bit, 8bit or binary.
I see you are suggesting validating and normalizing sub parts of message/* when user attaching eml, it is a bit too much for me. That requires a lot of efforts and still there will be eml files totally broken and can't be fixed. There can also be cases that we should not change headers of sub parts, because the original can't be restored on the receiving side.
Comment 21•3 years ago
|
||
(In reply to Ping Chen (:rnons) from comment #20)
(In reply to Alfred Peters from comment #14)
(In reply to Ping Chen (:rnons) from comment #11)
So we can't encode it with base64. Inserting line breaks may work in some cases, but I'm not sure.
I think you have misunderstood. [...]
I think there is no misunderstanding. The paragraph says for any MIME part, when Content-Type is multipart/* or message/*, the Content-Transfer-Encoding can only be 7bit, 8bit or binary.
Well, then I misunderstood you - sorry about that.
I see you are suggesting validating and normalizing sub parts of message/* when user attaching eml, it is a bit too much for me. That requires a lot of efforts and still there will be eml files totally broken and can't be fixed. There can also be cases that we should not change headers of sub parts, because the original can't be restored on the receiving side.
Yes, you are right, if the attachment itself is a multipart message, it will be difficult. However, it would be the cleanest solution.
I don't see the binary content type as a solution. Is this supported by any email reader at all? I have not encountered such a mail yet. But that doesn't have to mean anything either.
As a stopgap, we could display a warning, and leave it up to the user to decide whether to attempt to send, or to customize the email. Whereby probably the fewest are able to do so.
Comment 22•3 years ago
|
||
Sound like this is wontfix: no real solution in sight, for pretty low benefit even if a solution would be found.
Updated•3 years ago
|
Description
•